Social Media and Information Literacy

lable at ScienceDirect

Computers in Human Behavior 83 (2018) 278e287

Contents lists avai

Computers in Human Behavior

journal homepage: www.elsevier .com/locate/comphumbeh

Full length article

The diffusion of misinformation on social media: Temporal pattern, message, and source

Jieun Shin a, *, Lian Jian b, Kevin Driscoll c, François Bar b

a Department of Preventive Medicine, University of Southern California, 2001 N. Soto St., Los Angeles, CA 90032, USA b Annenberg School for Communication and Journalism, University of Southern California, USA c Department of Media Studies, University of Virginia, USA

a r t i c l e i n f o

Article history: Received 24 June 2017 Received in revised form 30 September 2017 Accepted 11 February 2018 Available online 13 February 2018

Keywords: Misinformation Rumor Social media Diffusion Partisan Election Fake news

* Corresponding author. E-mail address: [email protected] (J. Shin).

https://doi.org/10.1016/j.chb.2018.02.008 0747-5632/© 2018 Elsevier Ltd. All rights reserved.

a b s t r a c t

This study examines dynamic communication processes of political misinformation on social media focusing on three components: the temporal pattern, content mutation, and sources of misinformation. We traced the lifecycle of 17 popular political rumors that circulated on Twitter over 13 months during the 2012 U.S. presidential election. Using text analysis based on time series, we found that while false rumors (misinformation) tend to come back multiple times after the initial publication, true rumors (facts) do not. Rumor resurgence continues, often accompanying textual changes, until the tension around the target dissolves. We observed that rumors resurface by partisan news websites that repackage the old rumor into news and, gain visibility by influential Twitter users who introduce such rumor into the Twittersphere. In this paper, we argue that media scholars should consider the mutability of diffusing information, temporal recurrence of such messages, and the mechanism by which these messages evolve over time.

© 2018 Elsevier Ltd. All rights reserved.

1. Introduction

Prevalent misinformation online is a growing concern around the globe (WEF, 2014). Whether it is in the form of conspiracy theories or unsubstantiated rumors, false information is now a part of the contemporary media system where varying degrees of in- formation sources vie for our attention. In particular, “fake news,” which generally refers to fabricated news stories purporting to be true, came to the forefront in 2016, circulating wildly during the Brexit vote and the American presidential election. As a result, the Oxford English Dictionary named “post-truth” the 2016 word of the year to highlight less influential role of objective truth in shaping public opinions than political belief or emotion.

There has also been considerable research on this topic. Previous research investigated the effects of exposure to false information and corrections on attitudes and political behavior (Berinsky, 2017; Bode & Vraga, 2015; Cacciatore et al., 2014; Fridkin, Kenney, & Wintersieck, 2015; Garrett, 2011; Nyhan & Reifler, 2015; Uscinski, Klofstad, & Atkinson, 2016; Weeks & Garrett, 2014; Wood &

Porter, 2016). In general, these studies have found that in- dividuals are more likely to believe in dubious statements that match their partisanship than statements that run counter to their belief (e.g., Weeks, 2015). In addition, some studies reported that corrections usually work in experimental settings where in- dividuals are required to read random debunking messages (e.g., Nyhan & Reifler, 2015), although such efficacy was challenged in a social media environment where people selectively share correc- tive messages (Shin & Thorson, 2017).

Despite growing research in rumors and misinformation, what is largely missing from the current work is dynamic analysis of misinformation diffusion processes online. Scholars argue that misinformation gains its power when it is repeated and passed along from one person to another (DiFonzo& Bordia, 2007). That is, the defining characteristics of misinformation are its dynamic mode and collective process that unfolds over time. Therefore, unlike previous studies that examined misinformation as static communication and that take snapshots from experiments or sur- veys, we focus on changing communicative patterns that occur during the lifecycle of misinformation on social media. Largely exploratory in nature, this study examines a set of questions that shed new light on the nature of political rumoring e and diffusion of information broadly.

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287 279

Our study differs fromprevious research in the field of computer science and engineering (Bessi et al., 2015; Friggeri et al., 2014; Kwon, Cha, Jung, Chen, & Wang, 2014; Vicario et al., 2016), which places relatively less emphasis on understanding the social psy- chological context underlying the phenomenon. In addition, these previous studies tend to treat misinformation as if the diffusing message is a fixed object. On the other hand, our study takes an alternative perspective, which views misinformation to be mutable and malleable as they diffuse. We explore this idea using a multiple-case study approach, while paying attention to the distinct context of each rumor. Our research builds on studies (Allport& Postman,1947; Rojecki&Meraz, 2016) that exceptionally focused on dynamic communication process in the rumoring phenomenon. For instance, Rojecki and Meraz (2016) investigated the agenda setting power among webpages, Google searches, and media coverage with two rumor cases. Another classic study of rumor, conducted by Allport and Postman (1947), examined how rumor content changes in a serial transmission chain in which a story is passed along from one person to another.

Our study aims at investigating misinformation diffusion as an evolving phenomenon focusing on three components: temporal pattern, rumor narrative, and rumor sources. To achieve this goal, we employ various time series analysis on 17 political rumors that circulated on Twitter over 13 months during the 2012 U.S. election period. The context of the 2012 election on Twitter is still relevant today. First, Twitter emerged as a primary political communication channel during the 2012 election and still remains to be prominent (Conway, Kenski, & Wang, 2015). Second, political misinformation circulating on social media gained attention as a serious threat to democracy during 2012, and lately the phenomenon has become the center of public discussion (Ehrenberg, 2012; Shin, Jian, Driscoll, & Bar, 2016). Through our analyses, we show that many contested rumors resurface by partisanwebsites that repackage old rumors into “news”, and gain visibility by influential Twitter users who share such content with their followers. We also show that rumor resurgence often accompanies changes in content, generally in the direction of exaggeration, although this trend abruptly stops when the election is over. In this paper, we argue that digital media scholars should consider themutability of diffusing content and the mechanism by which messages change over time. We also high- light the underlying partisan media users’ motivations and strate- gies that drive the evolution of political misinformation.

2. Conceptual framework

2.1. Political misinformation in the internet age

Many different terms (listed in Table 1) such as misinformation, disinformation, and rumor are used interchangeably to describe information that lacks truth, despite their conceptual differences. For instance, both misinformation and disinformation highlight the state of information being untrue. However, the term misinforma- tion is agnostic regarding the motivation of falsehood, whereas disinformation assumes that inaccuracy stems from deliberate intention. Due to difficulties in identifying the intention of the

Table 1 Conceptual differences among terms referring to dubious claims.

Term Falsity Motivation

Disinformation False To gain an advantage Misinformation False Unknown Rumor Unknown Unknown Fake news False To gain an advantage Troll post Unknown To incite conflict

source, researchers often adopt the wordmisinformation to broadly describe false claims. On the other hand, rumor is largely defined as a piece of information that has not been confirmed (DiFonzo & Bordia, 2007). Therefore, a rumor may turn out to be true, even when it was not supported by concrete evidence at the time of circulation.

In addition, there are also terms such as trolling and fake news that focus on the source's deliberate motivation to provoke con- troversy and emotional responses online. Yet, the difference lies in whether such information is presented as a form of legitimate news (i.e., a researched fact) or a post (i.e., an opinion). Specifically, trolling is concerned about acts of posting offensive messages to online communities in order to incite conflict (Binns, 2012; Bishop, 2012; Coles & West, 2016), while fake news is about creating and disseminating false stories disguised as a credible news source for political or financial gain (Silverman, 2017; Vargo, Guo,& Amazeen, 2017). In addition, fake news is also different from troll posts in that fake news is almost always false or misleading, yet troll posts are not necessarily false. See review by Jack (2017) for other terms such as propaganda and gaslighting.

This genre of unsubstantiated claims proliferates online and has risen “to new levels of importance in this postmodern political context” (Harsin, 2012, p. 3). According to a recent Pew Research survey (2016), 32% of U.S. adults indicated that they often see completely made-up political stories, while that number goes up to 51% for somewhat inaccurate news. More troublingly, 23% have shared false stories with others either knowingly or not. While some research (Allcott & Gentzkow, 2017) has found that the impact of false news stories is only a fraction of television campaign ads, other studies showed that, nonetheless, the exposure to misinformation could have real consequences such as voting de- cisions (Weeks & Garrett, 2014) and mistrust in government (Einstein & Glick, 2015).

The current concern over political rumors and the like is deeply related to changing media environments (Harsin, 2012; Mocanu, Rossi, Zhang, & Karsai, 2015; Rojecki & Meraz, 2016). As much as digital media holds democratizing potential, it also has empowered individuals to plant and spread falsehood at a massive level. Today, anyonewith Internet access can participate in rumor spreading and influence the process with more or less power through various web applications (Mocanu et al., 2015; Simon, Goldberg, Leykin,& Adini, 2016). Once launched into the web, even seemingly preposterous claims can turn into a large cascade through networks of like- minded individuals and partisan organizations (Shin et al., 2016).

2.2. Temporal patterns of diffusion

Although challenging, examining diffusion patterns of infor- mation provides insight into the nature of the information. In particular, the shape of diffusion e e.g., bursty spreading patternse is an important indicator. For example, one of the mechanisms that give rise to a sudden burst of information cascade is the recency effect. In general, recent information is considered more news- worthy than older information and thus is more likely to be shared with others than older information (Xu, 2013). Such preference to

References

Faris et al. (2017); Jack (2017) Lewandowsky, Ecker, Seifert, Schwarz, and Cook (2012) Bordia & DiFonzo (2007); Rojecki & Meraz, 2016; Shin et al. (2016) Silverman (2017); Vargo et al. (2017) Binns (2012); Bishop (2012); Coles and West (2016)

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287280

newer information generates bursty and fast-paced information cycles, as attention moves onto the next new items quickly (Leskovec, Backstrom, & Kleinberg, 2009). Typically, the lifecycle of information is a matter of a few days, if not a few hours, showing the pattern of a sharp rise and fall (Kwak, Lee,&Moon, 2010; Nahon &Hemsley, 2013). Research shows that themajority of messages on Facebook and Twitter are shared within the first day of the original post (Bakshy, Rosenn, Marlow, & Adamic, 2012; Kwak et al., 2010).

However, not all types of information exhibit the same diffusion pattern. For instance, a recent study of popular image memes on Facebook (Cheng, Adamic, Kleinberg, & Leskovec, 2014) revealed that over half of viral memes came back in one or more subsequent bursts after a short or extended hiatus. In particular, the pattern of recurrence seems to be pronounced in the word-of-mouth type of rumors. For instance, in examining the diffusion patterns of general rumors such as the existence of Bigfoot, Kwon et al. (2014) found that these rumors tend to showmultiple and periodic spikes rather than a single spike. Friggeri et al. (2014) also found that general false rumors persisted even months after the original post and continued to flare up multiple times.

These previous findings pose further questions such as whether political rumors exhibit a certain temporal pattern. Unlike the Bigfoot rumor, which is a timeless and evergreen topic, political rumors are usually about a current political comment or unfolding situations. Thus, political rumors may show a single spike followed by gradual or rapid decline. Alternatively, political rumors may exhibit multiple repeated spikes, similar to evergreen rumors as long as the target of the rumor is relevant. Thus, we examine general temporal patterns of political rumors focusing on their recurrence. In addition, since our rumor collection contains both true rumors (i.e., facts) and false rumors (i.e., misinformation), we investigate whether there are any differences in the diffusion pat- terns between the two types of rumors.

R1a. Do political rumors recur?

R1b. Is there any difference in the diffusion patterns between true and false rumors?

2.3. Evolution of rumor content

Assuming that some rumors exhibit recurrence, we know little about how they come back. Traditional information diffusion scholars assumed that news content stays the same during the diffusion and thus paid little attention to message transformation (Im, Kim, Kim,& Kim, 2011). During the time when there were only a handful of powerful TV networks and daily newspapers that create information, this assumption may be justified. However, today's media environment is different, as there exists a complex information ecosystem where traditional media outlets and countless other types of sources co-produce and disseminate stories (Jenkins, 2006). In particular, non-traditional media (e.g., blogs) play an increasingly important role in creating a frame that deviates from mainstream media and influences the public's agenda (McCombs, 2013). Therefore, it is no longer realistic to as- sume that people are exposed to the same version of the news story throughout its lifetime.

As such, a recent study (Im et al., 2011) has shown that news stories indeed constantly evolve by adding new information or changing its narrative. The authors traced two (true) news stories, one originating from a news organization, and the other from a personal blog. They found that news originating from a blog devi- ated much more from the initial version of the story, while news originally published by a news website tended to stay unchanged during circulation. The authors suspected that the nature of content

(i.e., financial news vs. human-interest news) could be responsible for such differences, although they did not exclude other possibil- ities such as the role of initial source of the story (e.g., news orga- nization vs. blogger) in content transformation.

Similarly, rumors may also transform and evolve. The need for change is particularly present for those rumors that resurge mul- tiple times (hereafter refer to as “comeback rumors”). In general, people appear to share recently published news, rather than stories from previous news cycles. Then, one mechanism by which the same rumor can resurface at a later time is changing its frame or adding newdetails. In laboratory experiments, Allport and Postman (1947) observed that diffusing rumors often mutate, following the principle of (1) leveling, (2) sharpening, and (3) assimilation. Leveling refers to loss of details that are not interesting, while sharpening happens when certain elements are selected and pro- nounced. Assimilation describes the process in which stories pick up some new ideas that were not in the original to appeal to readers at the time of diffusion. These characteristics, in particular assimi- lation, are said to contribute to the persistence of rumor by making the storyline more attractive.

Yet, little research has been conducted on themutation of rumor content in a real-life setting where information does not neatly flow in one direction as seen in the study of Allport and Postman (1947). Therefore, we investigate whether rumors deviate from the original version of the story over time and if so, how rumors tend to change. We approach this question both quantitatively and qualitatively, using text similarity measure and visualization of rumor messages respectively.

RQ2. Do comeback rumors show content changes between peaks?

2.4. Rumor sources

Although the message-centered view above provides insight to semantic traits associated with persistent rumor, it carries a risk of downplaying the role of individuals who strategically alter the original information. For instance, Allport and Postman (1947)’s rumor transmission principles emphasize unconscious emotional and cognitive bias in people rather than an individual's strategic effort to change the story. On the other hand, the literature on political propaganda (Cunningham, 2002; Jowett & O'Donnell, 2014) often highlights the underlying motivation of information sources such as government, corporate, and media whose objective is to mobilize hatred against the enemy through various commu- nication techniques. Therefore, the source-centered analysis com- plements the message-centered view by revealing the designers of such messages.

In contrast to a large body of literature examining the charac- teristics of individuals who are prone to believing in dubious claims, few studies have investigated the sources of such claims. For instance, numerous studies have shown that an individual's pre- existing attitude toward the subject is a strong predictor of rumor belief (Ecker, Lewandowsky, Fenton, &Martin, 2014; Garrett, 2011). Prior studies also found that rumor acceptance is affected by other psychological factors such as high predispositions toward conspiratorial thinking (Uscinski et al., 2016) and anxiety about the current environment (Kwon & Rao, 2017; Weeks, 2015). However, there is little research examining the characteristics of rumor producers and their motivations.

Therefore, we focus on the sources that create and keep political misinformation afloat on social media. One way to identify sources of misinformation on Twitter is to investigate external links embedded in messages. Since there is a 140-character limit imposed by Twitter, users often add hyperlinks to other websites

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287 281

for providing further contextual information. The presence of external websites is particularly important in sharing of informa- tion that lacks factual basis because the link sharer can avoid re- sponsibility by revealing the original sources of information. Indeed, one experiment (Tanaka, Sakamoto, & Honda, 2014) found that participants were more likely to share questionable claims containing an URL with their friends than the same claims without an URL.

In this study, we investigate whether there is a constant stream of external webpages, which serve as a basis for rumor spreaders to rely on. Specifically, we trace the major rumor sources that are responsible for making the rumor popular at each peak. The web- sites that are known to propagate political rumors into social media include partisan news programs, alternative news media, and elite political blogs (Rojecki & Meraz, 2016; Vargo et al., 2017). Yet, it is also possible that the rumor phenomenon is largely an internal dynamic in which Twitter users depend on each other mutually for certifying false claims and accelerating the spread of rumor. If the latter were the case, rumor spreading would be self-sustaining without needing references to new external sources.

RQ3. Do persistent rumors depend on external sources that keep refreshing the rumor?

3. Method

3.1. Identification of rumors

This project focuses on political rumors (n¼ 17) that circulated on Twitter during the 2012 U.S. election (October 2011 to December 2012). The 17 rumors were a sub-section of the 57 rumor collec- tions, which we identified from three rumor-debunking websites: Factcheck.org, Snopes.com, and About.com's “Urban Legends” page. If any of these sites investigated a claim, we included it in our rumor collection regardless of whether the claim was true or false. For each of the 57 rumors, we collected related tweets in real time for 13 months (January 2012eJanuary 2013) using the Gnip Power- Track, which provided access to the Twitter firehose (access without rate limits). Unlike other streaming services, the firehose provides 100 percent of the publicly available tweets, along with metadata about the tweet (e.g., author of the tweet, time when the tweet was posted, and whether it was a retweet or an original tweet). When initially retrieving rumor related tweets from this dataset, we opted for a broad-match strategy that used the mini- mum number of keywords at the risk of retrieving many false positive tweets. Our goal was to identify as many tweets that were relevant to the rumor. For instance, the keyword combination of “obama” and “ring” and (“arabic” or “islamic” or “god” or “allah”) was used to retrieve tweets related to a rumor that Obama's wedding ring bore an inscription that says in Arabic “No god but Allah”.

Next, we hand-coded each tweet to remove messages that were not relevant to the specific rumor and to further classify rumor tweets into more fine-grained categories. Specifically, we asked four pairs of content coders to identify 1) whether each tweet was actually about the rumor and 2) if it was about the rumor, subse- quently whether attitude of the tweet was endorsing, rejecting, or unclear. Tweets repeating or confirming the rumor were coded as “endorsing.” Tweets denying the rumor or citing those who debunked the rumor were coded as “rejecting.” All other tweets were coded as “unclear.” For example, “Just saw the ring Obama has worn for over 30 years ‘there is no god except Allah’ vote wisely” was coded “endorsing”. “To the people emailing me about Obama's ‘shahada’ or ‘allah’ ring: you're actually crazy” was coded “reject- ing”. And “Obama's Allah ring stirs debate” was coded “unclear. .

We measured inter-coder reliability using Krippendorff's alpha. Reliability was checked at multiple points over time, since some rumors required several weeks of coding due to its large volume. Overall, reliability measures stayed above .75. For each rumor, disagreements were resolved by randomly adopting one of the values assigned by the two coders. The rumors that had more than 10,000 tweets were partially single coded. For these rumors (n¼ 48,442), we first randomly selected 1e5% of the entire tweets and content coded them. And only when inter-coder reliability was high (alpha >.8) for this initial set, the rest was singled coded. Of all the tweets preliminarily identified as relevant to these 57 rumors (n¼ 439,556) via keyword matching, 75.20% (n¼ 330,538) were confirmed as relevant by content coding.

For this project, we concentrated on rumors that were relatively popular (i.e., more than 3000 endorsing tweets) and whose life- cycles were completely captured within our data collection period. A total of 17 rumors (n¼ 274,416) qualified this rule. A detailed description of these 17 rumors is presented in Appendix 1. Since rejecting tweets were only a small percent, on average less than 3%, we focused on tweets that propagated the rumor rather than refuting it.

3.2. Recurrence: repetitive temporal pattern

Tweets within each rumor were arranged in chronological order starting from January 2012 to January 2013. Based on the daily volume of rumored tweets, we measured the signature of infor- mation diffusion using twomeasures: the number of spikes and the extent to which rumor tweets concentrated on the most active day. First, the number of spikes was automatically identified by the findpeaks function of the pracma package in R. This function returns an array of locations where the rumor peaked in the time series.We specified rules such that, if there is more than one peak, the dis- tance between peaks should be at least 7 days apart. This rule was to ignore small peaks that occur in the neighborhood of a large peak and thus identify a local maximum. In addition, we restricted a peak's height to be at least 10% of the largest peak to focus on ru- mors that were shared at a meaningful level. Second, as a com- plementary measure, we obtained the fraction of tweets in the most active day relative to the total volume. A higher value in- dicates higher concentration on the most active day.

3.3. Text similarity

For rumors with multiple peaks, we examined changes in rumor content both quantitatively and qualitatively. First, we identified messages that were representative of each peak day for each rumor. For this, we preprocessed tweets to focus on rumor relevant key- words and removed unnecessary languages such as username, emoticons, retweet markers (e.g., RT), punctuation marks, and stop words (e.g., prepositions, pronouns). Next, we transformed various words into their roots (e.g., goes / go), a process known as word stemming. Following this step, we broke each tweet intowords (i.e., tokens) and converted them to a document-term matrix where its rows represented the tweets (i.e., document), and columns corre- sponded towords (i.e., term) that appeared in that tweet. The value of each cell indicated the number of a particular word appearing in the given tweet. To draw our attention to important information, sparse words that occurred less than 10% of the entire corpus was removed.

Next, we measured the extent to which two corpuses of the same rumor is similar, using cosine similarity. This measure is amongst the most commonly used for quantifying similarities/ dissimilarities between texts. Cosine similarity indicates how close two rumor corpuses are in a multidimensional term-vector space,

Fig. 1. Example rumors, with a true rumor (red: rumor about Obama's daughter traveling to Mexico with secret agents) having a single peak, whereas a false rumor (green: rumor about Michelle Obama's comment on the flag) having multiple peaks. The aggregate sizes of the two rumors were similar: the one on the red had 4091 tweets, and the green one had 4790 tweets. However, a majority of activity related to the true rumor was concentrated in a single peak that was an order of magnitude greater than any one of the peaks related to for the false rumor. Bars are plotted next to each other rather instead of stacked on top of each other. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287282

expressed as the cosine of the angle between two collections of texts (See the formula below). In general, cosine similarity value is high when twomessages share same vocabulary and close together in term-vector space. A cosine similarity value of 1 implies that two texts are exactly the same, and a value of 0 implies complete dif- ference. Lastly, before we calculated cosine similarity, we normal- ized the frequency of terms to adjust for the different sizes of rumor corpuses.

Cosine ðcorpus1; corpus2Þ ¼ Pt

k¼1 � termik ; termjk;

� ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPt

k¼1ðtermik Þ2$ Pt

k¼1ðtermik Þ2 q

where termik and term termjk are the frequencies of word k in corpuses i and j.

For further guidance, the set of keywords on each peak day was also visualized as semantic networks. In each graph, nodes corre- spond to terms that frequently appear in the given rumor corpus, and edges indicate co-occurring relations. This visualization allows researchers to view a cluster of texts as a network of connected words and gather insight into how a certain idea or story is rep- resented (Drieger, 2013). In comparing semantic networks of rumor across peaks, we focused on the appearance of new words as well as the disappearance of words.

3.4. External rumor sources

External sources refer to websites to which tweets link for the rumor. We extracted hyperlinks (URLs) embedded in each tweet and produced a list of web domains sorted by frequency. For those shortened URLs such as ‘bit.ly’, we traced their final destinations to obtain distinct domains. In this analysis, we focused on the most frequently cited website from the collections of rumor across different peaks.

4. Results

The 17 rumors varied in their sizes ranging from the largest rumor close to 100,000 tweets (i.e., Romney campaign used the same slogan as the Ku Klux Klan e KKK ewhite supremacist or- ganizations) to the smallest one just being over 3000 (i.e., Obama campaign refused a prayer from a Catholic cardinal at the Demo- cratic National Convention e DNC). According to the analysis of the three fact-checking websites, 4 out of the 17 rumors were true, and the remaining 13 were false. True rumors, for example, included Obama's literary agency (accidently) listing Obama's birthplace as Kenya in a promotional booklet in 1991. False rumors included suspicion that Obama signed a total of 923 executive orders. Of these 17 rumors, the majority was about the presidential candi- dates from two rival parties in 2012: 12 rumors were about Barack Obama (candidate from the Democratic Party) and 4 about Mitt Romney (candidate from the Republican Party). There was one rumor for Rick Santorum (Republican primary candidate). Notably, all of the 17 rumors were negative toward the target. Appendix 1 gives a detailed description of each rumor.

Additionally, we traced the first appearance of rumor source in our Twitter dataset and found that 5 rumors originated from traditional media, while 12 rumors originated from non-traditional media. Here, traditional media is defined by sources archived by the LexisNexis database, which documents 13,818 mainstream news sources globally. Specifically, we found that 3 of the 4 true rumors (i.e., facts) had traditional media as origin such as CNN and AP, whereas 9 of the 11 false rumors (i.e., misinformation) originated from non-traditional media including satirical websites, partisan

news websites, YouTube videos, and blogs. Lastly, we found that the spreaders of the Obama-related ru-

mors (negative rumors about Obama) were overwhelmingly more Republicans than Democrats (84% vs. 16%), while the spreaders of the Romney-related rumors (negative rumors about Romney) were mainly Democrats (91% vs. 9%). A detailed explanation of the method is presented in Appendix 2.

4.1. Temporal patterns

RQ1 asked whether political rumors recur, and if so, whether there were any differences between true rumors and false rumors in their temporal patterns. First, we found that 11 out of 17 rumors came back, exhibiting multiple peaks. However, close examination revealed that this pattern was only applicable to false rumors. False rumors (n¼ 13) had an average of 3.31 peaks (SD¼ 1.60), while all of the true rumors (n¼ 4) had a single peak (SD¼ 0). This difference was statistically significant (Wilcoxon rank-sum test, p< 0.05; two sided). This means that false rumors tend to flare up again, when true rumors do not. In particular, 5 out of the 11 comeback rumors erupted on the Election Day or the day before. For instance, the Romney-KKK rumor, whichwas first propagated by a blogger in late 2011, went viral on September 3, 2012 (n¼ 16,530), long after its initial public appearance. This rumor seemed to sharply decline, and yet came back with an even bigger resurgence in two months, exactly on the Election Day (n¼ 16,705). Similarly, a rumor that Michelle Obama said “all this for a damned flag” during a 9/11 ceremony was brought back 5 times on days like the National Flag Day (June 14, 20012), 9/11 Memorial Day (September 11, 2012), and the day before Election Day (November 5, 2012). In contrast, true rumors, such as Obama's daughter Malia traveling to Mexico on Spring break with secret agents, did not resurge after the initial peak (see Fig. 1).

Two exceptional cases in which false rumors did not recur were a claim about Obama wearing an Arabic ring (first reported by a right-wing news site, WND) and a claim about Obama refusing a prayer from a Catholic Cardinal at DNC (first propagated by a blogger). Of these, the rumor concerning Obama's refusal stopped

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287 283

circulating when Cardinal Dolan actually gave a prayer and thus outright contradicted the rumor.

This finding was consistent with the analysis of tweet volume distribution, which showed burstiness for true rumors. We found that true rumors had half of their tweets (Mean¼ 49.58%) posted on the most active day relative to the entire volume for that rumor. In contrast, false rumors had only 18.57% on one peak day. This confirms that the true rumors in our data show a single prominent spike in their temporal features, whereas the false rumors show repeated peaks over an extended period of time. For example, a rumor that exhibited the highest level of burstiness (61.06%) was a true claim about Obama's literary agent (accidentally) listing Oba- ma's birthplace as Kenya. A rumor with the lowest level of bursti- ness (3.42%) was a false claim about Michelle Obama's comment on the American flag, which resurged 5 times in 2012.

In addition, a visual inspection of data over time revealed that the rumors about Obama and Romney stopped spreading after the Election Day. Therefore, we conducted time series analyses to es- timate the difference in the trend of rumor volume 60 days before and after the Election Day. As presented in Fig. 2, we found that there was a significant immediate effect on the trend of daily rumor counts following the Election Day such that both Obama rumors (n¼ 12) and Romney rumors (n¼ 4) volume fell immediately when the election was over. After the election, the trend was essentially flat indicating disappearance of the rumors. Similarly, one rumor targeting Santorum gradually disappeared around the time (April 2012) he ended his campaign for presidency.

4.2. Evolution of rumor content

So far we have found that false rumors tend to come back when true rumors do not. Now, we proceed to explore whether the content of comeback rumor changes over time or remains the same (RQ2). For this research question, we drew on both quantitative (i.e., similarity measure) and qualitative approaches (i.e., visuali- zation of semantic network). Note that we analyzed only 11 comeback rumors (all false) excluding 6 rumors that had a single peak. Using text similarity analyses of the same rumor across

Fig. 2. Daily rumor counts of 12 Obama rumors combined (left) and 4 Romney rumors combined (right) 60 days before and after the Election Day. Dotted vertical line in- dicates the Election Day. There was a significant drop in the number of rumor tweets for both Obama and Romney rumors (Obama: B¼�734.76, p< 0.05; Romney: B¼�4215.68, p< 0.01).

multiple peaks, we found that 6 out of the 11 rumors had their average cosine values less than 0.5, indicating some differences among rumor content. To be clear, there is no single cut-off value to identify when two texts are similar or dissimilar. The value of cosine similarity ranges from 0 to 1, and a threshold may vary within this range depending on the context and the purpose of analysis. In this study, we used an arbitrary threshold of 0.5, which is often used as default in many other studies (e.g., Jansen & Van Durme, 2011; Lam, Sleeman, & Vasconcelos, 2005). A close exam- ination of semantic rumor networks also revealed that this threshold served as a reasonable baseline, as rumors with cosine value less than 0.5 accompanied considerable changes in content by picking up new concepts or dropping previously presented keywords.

For instance, the rumor that showed the highest similarity measure (alpha¼ 0.82) was the Romney-KKK rumor. Originated from a blog post, this rumor was represented by the following set of keywords consistently throughout three peaks: “That awkward moment that Mitt Romney's campaign slogan ‘Keep America American’ was the same slogan used by the KKK in 1922” (bold indicating keywords contained in the network visualization). Although the rumor showed a minor spelling correction from an earlier version to a later version (“akward” -> “awkward”), this rumor kept its format almost unchanged for over a year. Another rumor whose content changed minimally was the claim of Romney saying that “I can relate to black people, because my ancestors owned slaves.” This rumor was produced from a satirical website. It spread over 8 months with its message frame relatively intact. This rumor contained a quotation, which may be one of the reasons why this rumor showed small amounts of textual changes.

On the other hand, the rumor that had the lowest average similarity measure (alpha¼ 0.30) during the diffusion process was that “Obama has signed 923 executive orders.” This rumor changed its frame at each peak by adding a different reference such as: Obama has signed 923 executive orders and played over 1500 holes of golf / Obama has signed 923 executive orders in 40 months / Obama has signed 923 executive orders in 3.5 years, Nixon 1 in 6, Carter 3 in 4, Reagan 5 in 8, Bush in 3 in 4, Clinton 1 in 6, GW Bush 82 in 8 / Obama has issued more executive orders then all other Presidents combined. 923 in just 3.5 years / 923 executive orders by Obama in 3.5 years 800 times more than any other president (See Fig. 3 for an example of visualization of this rumor). In addi- tion, the claim of the Obama administration planning to ban guns for U.S. citizens through the United Nations (UN) showed a rela- tively high level of distortion (alpha¼ 0.30). This rumor turned into a message that urged people to buy a gun right now because the Obama administration would regulate guns through the UN.

We also qualitatively observed one consistent pattern in the comeback rumors' diffusion: false rumors turned into a more intense and extreme version over time. For instance, the rumor about Romney's family owning a voting-machine company started as a mere allegation that was worthy of “investigation” and that could “possibly go wrong (bold indicating terms that appeared in the corpus).” However, this rumor ended with a version in which voting machines in Ohio actually malfunctioned in favor of Romney on Election Day. Similarly, the rumor of Obama signing 923 exec- utive orders became more extreme over time by using an exag- gerated reference. At the end of the diffusion, which was one week before Election Day, the extent of Obama's executive orders turned into “800 times more than any other president.” Another rumor involvingMichelle Obama seen in a video saying “all this for a damn flag” transformed into a rumor that such incident had been confirmed by lip readers. Moreover, rumors tended to picked up more extreme adjectives and modifiers such as “very”, “much”, “indeed”, and “urgent” in a later stage of diffusion.

Fig. 3. Visualization of the rumor regarding Obama's executive orders in two periods of time. Red color indicates words appearing in both periods. Blue color indicates words (e.g., golf, hole) showing up only in the rumor left (June 11, 2012), while yellow color indicates words (e.g., Bush, Nixon) appearing only in the rumor on the right (September 29, 2012). (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287284

4.3. External sources of rumor

RQ3 asked whether comeback rumors depended on external sources that fed the rumor, or whether they were independent of such external sources. Overall, we found that more than 70% of rumor tweets contained a hyperlink to external websites. We also found that the most frequently cited website changed consistently from peak to peak. Specifically, for 7 out of 11 comeback rumors, each peak featured a different website as a major source of the rumor. Most of these sites were non-traditional media outlets including so-called fake news, although these sources also included occasional mainstream media outlets. The non-traditional news websites tended to follow up on the old rumor with a slightly different angle and a recent timestamp, making the rumor appear as new information. In general, while the rumors negative to Obamawere followed up by right-wingmedia, the rumors negative to Romney were repeated by left-wing media. For example, the rumor about Obama's sealed records returned five times with each peak linking to several small-scale conservative political news sites.

Exceptionally, there were 3 rumors that did not depend on a stream of different external sources for resurgence. For instance, 87% of the Michelle-Flag rumor tweets contained a link to the same YouTube video over 5 peaks during the entire diffusion process. Additionally, we found that links to external sources disappeared for two rumors, as they propagated over extended period of time. Both the Romney-KKK rumor and Romney-Slave rumor were two such cases. The Romney-KKK rumor tweets initially cited a blog post that first wrongly accused Romney of using KKK slogan. However, since major news outlets (e.g., MSNBC and the Wash- ington Post) publicly apologized for their coverage of the rumor based on a blog post without checking its veracity, the rumor turned into some kind of joke in which Twitter users shared a humorous text. Similarly, the Romney-Slave rumor, which was produced by the satire website (freewoodpost.com), showed disappearance of citations over time. In the first two peaks, 66% of the tweets about this rumor contained a link to the satire site, yet the presence of the hyperlink fell below 5% in the later three peaks.

In particular, when we closely examined these 3 comeback ru- mors, which were independent of external websites that fed fresh information, we observed that each recurrence was attributable to highly visible Twitter users. These users tweeted a rumor without citing other websites for further information. In 11 out of 13 peaks for these rumors, the most frequently shared messages were from those with more than 3400 followers (i.e., top 1% users). For instance, three peaks of the Romney-KKK rumor were dominated by retweets of high profile accounts such as a Seth Rogen

(Hollywood actor) parody account, a news aggregate account for college students, and an Obama fan account, respectively.

Additionally, we found that there was a small overlap (on average 4%) in Twitter users between different peaks for the comeback rumors. Most of these overlapping users were not power users themselves, but appeared to have played a role in diffusing the rumor into different pockets of Twitter users by repeating or updating the old rumor. Indeed, the rumor may have appeared new to the majority of users in each peak.

5. Discussion

In this study, we traced the lifecycle of 17 popular political ru- mors that circulated in 2012 on Twitter by sifting through Twitter's enormous haystack of information. Previous research examining the phenomenon of political rumor and misinformation has largely missed this dynamic diffusion process such as how false informa- tion emerges, declines, and recurs on social media. Additionally, prior studies generally assumed that diffusing information remains unchanged in terms of its frame or details during circulation (Im et al., 2011). Thus, little is known about whether or how the mes- sage containing misinformation changes during its lifecycle and what roles individual media users play in this process. To fill this void, we investigated the dynamic diffusion process of misinfor- mation focusing on its temporal pattern, content change, and in- formation source.

First, we found that most false rumors repeated periodically, whereas true rumors did not. Specifically, our data showed that 11 out of 13 false rumors resurged multiple times after the initial burst, while true rumors showed a single spike of sharing without a comeback at a later time. This pattern may mean that rumor spreaders strategically bring back false rumors in hopes of influ- encing others. In particular, we observed many of these rumors resurge nearing the Election Day and yet they stopped spreading abruptly after the Election Day. These findings suggest that the political misinformation phenomenon could be a reflection of campaign tactics employed by those media professionals and in- dividual activists that Rojecki and Meraz (2016) call “partisan en- trepreneurs” ewho seek political power through the manipulation of information.

Then the question becomes why rumormongers do not resus- citate true rumors, which are equally negative toward the target politician. Wouldn't true rumors be more effective and less risky to repeat for political gains? We have three possible explanations. First, rumor spreaders intuitively focus on false rumors, because they believe that those false ones need more “promotion” or top-

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287 285

down effort, whereas true rumors do not need their help due to the presence of clear evidence. Second, another promising explanation may be the nature of rumor origin. While true rumors often orig- inate from mainstream news outlets (3 out of 4 rumors in our dataset), most false rumors are from relatively obscure websites. Hence, it is possible that rumor spreaders perceive stories origi- nating from major websites to have already exhausted their po- tential readers, yet perceive that stories with low initial exposure need further distribution. Although our current data does not allow us to examine the independent effect of two factors (rumor falsity vs. rumor origin) on rumor pattern due to the small sample size, this area of study certainly warrants future investigation. Lastly, sharing controversial rumors may serve a distinct identity- signaling function rather than a persuasion function. Since belief in false rumors is determined by partisanship, not by external validation, participating in a common epistemological sphere strengthens bonding within partisan networks and creates group solidary more so than true rumors.

To further examine the comeback rumors at the message level, we conducted quantitative and qualitative text analyses on the same rumor across different time periods. First, we found that old rumors often returnwearing a slightly newoutfit such as a different focus or new details. For instance, a claim that “a controversial rumor has been confirmed” was a typical strategy for rumor pub- lishing websites to add value to the old rumor. However, this does not necessarily mean that the comeback rumors change narratives only to make an appeal for existing audiences who are already familiar with the old rumor. The fact that there were only a small number of users who consistently followed through the same rumor suggests that evolving rumors moves onto different pockets of new audiences. Therefore, major changes in rumor content may be the results of the editorial process undertaken by media pro- fessionals or political groups who perceive the need to make the story more attractive.

Furthermore, we calculated the extent to which rumor mes- sages change across different time periods using the cosine text similarity measure. This quantitative text analysis confirmed that the rumor content indeed changed, showing different amounts of mutations across the rumors. For example, a rumor about Obama signing 923 executive orders showed the highest level of mutation. This rumor transformed into a different narrative for each of the 5 peaks, shifting its focus from him abusing his power while playing over 1500 holes of golf to his executive orders 800 times more than any other president's.

Overall, these patterns in rumor change are consistent with the tendencies that Allport and Postman (1947) found in their story- telling experiments. They argue that a story becomes leveled (boring details filtered out), sharpened (a focal point exaggerated), and assimilated (modified to reflect the individual's motivation and stereotype toward the target). Consistently in our dataset, we found that rumor storylines generally became exaggerated over time and turned aggressive. We also observed that these rumors were adapted to piggyback on the important issue of themoment such as inclusion of a trending hashtag (e.g., #flagday) or an environmental trigger (e.g., the Democratic National Committee), which is an indication of assimilation. Rumor tweets also picked up stronger adjectives (e.g., indeed, urgent, very) and more partisan hashtags (e.g., #tcot, #p2, #ccot) over time. Future research may focus on specific dimensions of content change such as sentiment (e.g., negative, positive), text length, and narrative coherence.

Lastly, we investigated the engines and drivers of the rumors. Focusing on external websites to which rumor tweets refer, we found that false rumors were mostly driven by non-traditional news websites who followed up on the old rumors. Specifically, we found that 7 out of 11 comeback rumors featured various

different websites as a major source at each peak over the entire lifecycle. Some of these websites e especially those that are often labeled as fake news today e appeared in several different rumors at different time points. This indicates that there is a group of “rumor entrepreneurs” who not only produce false claims but also give life back to old debunked rumors. However, we observed that thesewebsites were not alone responsible for generating the rumor phenomena. Rumor content usually gains visibility through Twitter users with large followers who link such content in their tweet, creating a viral event even several months after the initial publi- cation of the rumor.

The fact that false rumors are repeated multiple times and fol- lowed up by a number of different sources has implications for debunking misinformation. Previous research (Centola & Macy, 2007) has shown that receiving social reinforcement from multi- ple contacts significantly increases the likelihood that the person believes in themessage and takes action. Moreover, ameta-analysis of studies on message repetition (Dechene, Stahl, Hansen, & Wanke, 2010) revealed that merely repeating a message makes the story more familiar and thus more credible than a message told once. These findings suggest that collectively repeated misinfor- mation and false rumors e as seen in our study e may become extremely difficult to dislodge because they create the “illusion of truth effect” which refers to increasing power of repeated state- ments in believability (Silva, Garcia-Marques, & Reber, 2017).

At a more practical level, our findings suggest that detecting political misinformation is more challenging than it may seem since the sources and messages evolve over time. For instance, the list of suspicious websites may never be complete due to a constant stream of new players. However, the finding that false rumors tend to come back multiple times and become exaggerated at a later time can be useful for media literacy programs to combat misin- formation. Recently, researchers (Cook, Lewandowsky, & Ecker, 2017; van der Linden, Leiserowitz, Rosenthal, & Maibach, 2017) found that wide spread of fake news can be prevented if people are warned in advance that they will be exposed to false claims. This “psychological vaccine” effect may be particularly effective when we know which claims and when they are likely to appear again.

Additionally, based on our findings, it is recommended that fact- checking organizations should repeat their debunking messages when the debunked false rumors come back. Once a rumor is debunked, fact-checking organizations tend to move onto new rumors and do not revisit the old rumor. Instead, fact-checking practitioners could reverse-engineer the strategy that works for fake news and the spread of misinformation by sharing the previ- ously debunked rumors on social media when such claims resurface.

Exploratory in nature, our study has a number of limitations. The purpose of this study was to identify diffusion patterns of misinformation throughout its lifecycle, using a small number (albeit large volume) of political rumor cases (n¼ 17). Therefore, our findings may be limited in terms of scope and inferences, as they could be partially a consequence of the characteristics of our sample. Nevertheless, the strength of this study is that we accu- rately traced large-scale rumors (e.g., a rumor larger than 96,000 tweets) over an extended period of time (i.e., 13 months) and provided a foundation for developing future research. In addition, our study is only a starting point to systematically measuring tex- tual changes over time. In this paper, we treated words as the basic unit to represent the text and calculated text similarity using the cosine measure. As different units (e.g., n-gram) andmeasures (e.g., Euclidean) can yield different results, future research may also consider alternative approaches to measuring textual changes. Lastly, our dataset may contain automated bots set up by political or commercial groups. It is unclear what roles these bots e if there

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287286

were anye have played in the rumor diffusion and how they biased our results. However, our analysis of verified power users and external websites offer a consistent story in terms of temporal patterns and content changes reported earlier.

In sum, this study examined the diffusion of political misinfor- mation on Twitter. Along with cyberbullying (Sabella, Patchin, & Hinduja, 2013) and trolling (Binns, 2012; Bishop, 2012), misinfor- mation circulating online is an emerging problem that demands further attention. Our analysis showed that while partisan news sites and clickbait sites lent impetus to a rumor cascade and shaped the narrative of the rumor, social media users gave visibility to rumor content by sharing hyperlinks to the rumor with their fol- lowers. This pattern was repeated, each time with a slightly different storyline, until the tension around the target dissipated. In particular, the periodic recurrence was a signature for false rumors originating from obscure websites. As such, focus on the lifecycle analysis of rumor as opposed to snapshot analysis from experi- ments or surveys offers a promising approach for understanding political rumoring phenomenon.

Appendix A. Supplementary data

Supplementary data related to this article can be found at https://doi.org/10.1016/j.chb.2018.02.008.

References

Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. National Bureau of Economic Research. Retrieved from http://www.nber.org/ papers/w23089.

Allport, G. W., & Postman, L. (1947). An analysis of rumor. Public Opinion Quarterly, 10(4), 501e517. https://doi.org/10.1093/poq/10.4.501.

Bakshy, E., Rosenn, I., Marlow, C., & Adamic, L. (2012). The role of networks in in- formation diffusion. In Proceedings of the 21st international conference on world wide web. ACM.

Berinsky, A. J. (2017). Rumors and health care reform: Experiments in political misinformation. British Journal of Political Science, 47(2), 241e262.

Bessi, A., Coletto, M., Davidescu, G. A., Scala, A., Caldarelli, G., & Quattrociocchi, W. (2015). Science vs. conspiracy: Collective narrative in the age of misinformation. PLoS One, 10(2). https://doi.org/10.1371/journal.pone.0118093.

Binns, A. (2012). Don't feed the trolls! Managing troublemakers in magazines' online communities. Journalism Practice, 6, 547e562.

Bishop, J. (2012). The psychology of trolling and lurking: The role of defriending and gamification for increasing participation in online communities using seductive narratives. In H. Li (Ed.), Virtual community participation and moti- vation: Cross- disciplinary theories (pp. 160e176). Hershey, PA: Information Science Reference.

Bode, L., & Vraga, E. K. (2015). Correction of misinformation through related stories functionality in social media. Journal of Communication, 65(4), 619e638.

Cacciatore, M. A., Yeo, S. K., Scheufele, D. A., Xenos, M. A., Choi, D. H., Brossard, D.,…Corley, E. A. (2014). Misperceptions in polarized politics: The role of knowledge, religiosity, and media. Political Science & Politics, 47(3), 654e661.

Centola, D., & Macy, M. (2007). Complex contagions and the weakness of long ties. American Journal of Sociology, 113(3), 702e734.

Cheng, J., Adamic, L., Dow, P. A., Kleinberg, J. M., & Leskovec, J. (2014 April). Can cascades be predicted?. In In proceedings of the 23rd international conference on World Wide Web (pp. 925e936). ACM.

Coles, B. A., & West, M. (2016). Trolling the trolls: Online forum users constructions of the nature and properties of trolling. Computers in Human Behavior, 60, 233e244.

Conway, B. A., Kenski, K., & Wang, D. (2015). The rise of Twitter in the political campaign: Searching for intermedia agenda-setting effects in the presidential primary. Journal of Computer-Mediated Communication, 20(4), 363e380.

Cook, J., Lewandowsky, S., & Ecker, U. (2017). Neutralizing misinformation through inoculation: Exposing misleading argumentation techniques reduces their in- fluence. PLoS One, 12(5).

Cunningham, S. B. (2002). The idea of propaganda: A reconstruction. Westport, CT: Greenwood Publishing Group.

Dechene, A., Stahl, C., Hansen, J., & Wanke, M. (2010). The truth about the truth: A meta-analytic review of the truth effect. Personality and Social Psychology Re- view, 14(2), 238e257.

Difonzo, N., & Bordia, P. (2007). Rumor psychology: Social and organizational ap- proaches. Washington, DC: American Psychological Association.

Drieger, P. (2013). Semantic network analysis as a method for visual text analytics. Procedia-social and behavioral sciences, 79, 4e17.

Ecker, U. K., Lewandowsky, S., Fenton, O., & Martin, K. (2014). Do people keep

believing because they want to? Preexisting attitudes and the continued in- fluence of misinformation. Memory & Cognition, 42(2), 292e304.

Ehrenberg, R. (2012). Social media sway: Worries over political misinformation on Twitter attracts scientists' attention. Science News, 182(8), 22e25.

Einstein, K. L., & Glick, D. M. (2015). Do I think BLS data are BS? The consequences of conspiracy theories. Political Behavior, 37(3), 679e701.

Faris, R., Hal, R., Etling, B., Bourassa, N., Zuckerman, E., & Benkler, Y. (2017). Parti- sanship, propaganda, and disinformation: Online media and the 2016 U.S. Presi- dential election. Berkman Klein Center. Available at: SSRN: https://ssrn.com/ abstract¼3019414.

Fridkin, K., Kenney, P. J., & Wintersieck, A. (2015). Liar, liar, pants on fire: How fact- checking influence citizens' reactions to negative advertising. Political Communication, 32(1), 127e151.

Friggeri, A., Adamic, L. A., Eckles, D., et al. (2014). Rumor cascades. In Proceedings of the eighth international AAAI conference on weblogs and social media (pp. 101e110).

Garrett, R. K. (2011). Troubling consequences of online political rumoring. Human Communication Research, 37(2), 255e274.

Harsin, J. (2012). Attention! Rumor bombs, affect, managed democracy 3.0. Available at: SSRN: https://ssrn.com/abstract¼2138601.

Im, Y. H., Kim, E. M., Kim, K., & Kim, Y. (2011). The emerging mediascape, same old theories? A case study of online news diffusion in Korea. New Media & Society, 13(4), 605e625. https://doi.org/10.1177/1461444810377916.

Jack, C. (2017). Lexicon of lies : Terms for problematic information. Data & Society Publication. Retrieved from https://datasociety.net/pubs/oh/DataAndSociety_ LexiconofLies.pdf.

Jansen, A., & Van Durme, B. (2011 December). Efficient spoken term discovery using randomized algorithms. In Automatic speech recognition and understanding (ASRU), 2011 IEEE workshop (pp. 401e406).

Jenkins, H. (2006). Convergence culture: Where old and new media collide. New York: New York University Press.

Jowett, G. S., & O'Donnell, V. (2014). Propaganda & persuasion. Washington, DC: Sage.

Kwak, H., Lee, C., & Moon, S. (2010). What is Twitter, a social network or a news media. In Proceedings of the 19th world-wide web (WWW) conference, Raleigh, NC, 26e30 April.

Kwon, S., Cha, M., Jung, K., Chen, W., & Wang, Y. (2014). Prominent features of rumor propagation in online social media. In Data mining (ICDM). IEEE 13th interna- tional conference on IEEE. December 2013.

Kwon, K. H., & Rao, H. R. (2017). Cyber-rumor sharing under a homeland security threat in the context of government Internet surveillance: The case of South-North Korea conflict. Government Information Quarterly. https://doi.org/10.1016/ j.giq.2017.04.002. Advance online publication.

Lam, S., Sleeman, D., & Vasconcelos, W. (2005). Retaxþ: A cooperative taxonomy revision tool. Application and innovations in intelligent systems XII (pp. 64e77). London: Springer.

Leskovec, J., Backstrom, L., & Kleinberg, J. (2009 June). Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 497e506). ACM.

Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misin- formation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106e131.

van der Linden, S., Leiserowitz, A., Rosenthal, S., & Maibach, E. (2017). Inoculating the public against misinformation about climate change. Global challenges, 1(2). https://doi.org/10.1002/gch2.201600008.

McCombs, M. (2013). Setting the agenda: The mass media and public opinion. Cam- bridge: Polity Press.

Mocanu, D., Rossi, L., Zhang, Q., Karsai, M., & Quattrociocchi. (2015). Collective attention in the age of (mis)information. Computers in Human Behavior, 51, 1198e1204.

Nahon, K., & Hemsley, J. (2013). Going viral. Malden, MA: Polity. Nyhan, B., & Reifler, J. (2015). Estimating fact-checking’s effects: Evidence from a long

term experiment during campaign 2014. Retrieved at http://www. americanpressinstitute.org/wp-content/uploads/2015/04/Estimating-Fact- Checkings-Effect.pdf.

Rojecki, A., & Meraz, S. (2016). Rumors and factitious information blends: The role of the web in speculative politics. New Media & Society, 18(1), 25e43.

Sabella, R. A., Patchin, J. W., & Hinduja, S. (2013). Cyberbullying myths and realities. Computers in Human Behavior, 29(6), 2703e2711.

Shin, J., Jian, L., Driscoll, B., & Bar, F. (2016). Political rumoring on Twitter during the 2012 US presidential election: Rumor diffusion and correction. New Media & Society.

Shin, J., & Thorson, K. (2017). Partisan selective sharing: The biased diffusion of fact- checking messages on social media. Journal of Communication, 67(2), 233e255.

Silva, R. R., Garcia-Marques, T., & Reber, R. (2017). The informative value of type of repetition: Perceptual and conceptual fluency influences on judgments of truth. Consciousness and Cognition, 51, 53e67.

Silverman, C. (2017). What exactly is fake news? The fake newsletter, 26 Feburary. Retrieved from http://us2.campaign-archive1.com/?u¼657b595bbd3c63 e045787f019&id¼e0b2b9eaf0&e¼30348b6327.

Simon, T., Goldberg, A., Leykin, D., & Adini, B. (2016). Kidnapping WhatsApp- Ru- mors during the search and rescue operation of three kidnapped youth. Com- puters in Human Behavior, 64, 183e190.

Tanaka, Y., Sakamoto, Y., & Honda, H. (2014). The impact of posting URLs in disaster- related tweets on rumor spreading behavior. In Proceedings of the 47th Hawaii

J. Shin et al. / Computers in Human Behavior 83 (2018) 278e287 287

international conference on system sciences (HICCS-47) (pp. 520e529). Uscinski, J. E., Klofstad, C., & Atkinson, M. D. (2016). What drives conspiratorial

beliefs? The role of informational cues and predispositions. Political Research Quarterly, 69, 57e71.

Vargo, C. J., Guo, L., & Amazeen, M. A. (2017). The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014-2016. New Media & Society. https://doi.org/10.1177/1461444817712086. Advance online publication.

Vicario, M. D., Bessi, A., Zollo, F., et al. (2016). The spreading of misinformation online. Proceedings of the National Academy of Sciences of the United States of America, 113(3), 554e559.

Weeks, B. E. (2015). Emotions, partisanship, and misperceptions: How anger and anxiety moderate the effect of partisan bias on susceptibility to political

misinformation. Journal of Communication, 65(4), 669e719. Weeks, B. E., & Garrett, R. K. (2014). Electoral consequences of political rumors:

Motivated reasoning, candidate rumors, and vote choice during the 2008 U.S. Presidential election. International Journal of Public Opinion Research, 26(4), 401e422.

Wood, T., & Porter, E. (2016). The elusive backfire effect: Mass attitudes' steadfast factual adherence. Available at: https://ssrn.com/abstract¼2819073.

World Economic Forum. (2014). The rapid spread of misinformation online. Available at: http://reports.weforum.org/outlook-14/top-ten-trends-category-page/10- the-rapid-spread-of-misinformation-online/.

Xu, Q. (2013). Social recommendation, source credibility and recency: Effects of news cues in a social bookmarking website. Journalism & Mass Communication Quarterly, 90(4), 757e775.