ref
British Journal of Management, Vol. 33, 1238–1253 (2022) DOI: 10.1111/1467-8551.12554
Social Bots and the Spread of Disinformation in Social Media: The Challenges of Artificial Intelligence
Nick Hajli ,1 Usman Saeed,2 Mina Tajvidi 3 and Farid Shirazi 4
1School of Management, Swansea University, Swansea, SA2 8PP, UK, 2Data Science Lab, Ryerson University, Toronto, M5B 2K3, Canada, 3Department of Marketing, Queen Mary University of London, London, E1 4NS, UK, and 4Ted Rogers School of Information Management, Ryerson University, Toronto, M5B 2K3, Canada
Corresponding author email: f2shiraz@ryerson.ca
Artificial intelligence (AI) is creating a revolution in business and society at large, as well as challenges for organizations. AI-powered social bots can sense, think and act on social media platforms in ways similar to humans. The challenge is that social bots can perform many harmful actions, such as providing wrong information to people, escalating argu- ments, perpetrating scams and exploiting the stock market. As such, an understanding of different kinds of social bots and their authors’ intentions is vital from the management perspective. Drawing from the actor-network theory (ANT), this study investigates hu- man and non-human actors’ roles in social media, particularly Twitter.We use textmining and machine learning techniques, and after applying different pre-processing techniques, we applied the bag of words model to a dataset of 30,000 English-language tweets. The present research is among the few studies to use a theory-based focus to look, through ex- perimental research, at the role of social bots and the spread of disinformation in social media. Firms can use our tool for the early detection of harmful social bots before they can spread misinformation on social media about their organizations.
Introduction
Existing studies emphasize the value of social networks for firms and people (Candi et al., 2018; Leonardi, 2017; Mangold and Faulds, 2009; Sigfusson and Chetty, 2013; Williams, Du and Zhang, 2020). Product reviews through social media, for example, are becoming a rich source of consumer information (Moon and Kamakura, 2017). However, other research highlights social media’s potentially harmful consequences for pub- lic discourse (Miranda, Young and Yetgin, 2016). Information produced by individuals on social networking sites (SNSs), for example, has ethical consequences for business, such as spreading mis- information and compromising privacy, security and trust (Nadeem et al., 2019; Wang et al., 2019). With new artificial intelligence (AI) advancements,
particularly the emergence of deep machine learn- ing, we face new challenges; despite the opportu- nities AI presents for firms (Ross et al., ). Social bots within SNSs are autonomous actors driven by algorithms and software that post content on these platforms.Malicious bots are developed with the intention to harm. Research shows that these types of bots deceive and manipulate the stock market and also influence social media dialogues with fake news and misinformation (Kudugunta and Ferrara, 2018; Shi, Zhang and Choo, 2019).
Twitter has about 23 million social bots, ac- counting for 8.5% of total users (Lima Salge and Berente, 2017). In addition, over two-thirds of tweets come from social bots. A 2018 Pew research study examined 1.2 million English-language tweets over a period of 47 days. The result showed that 66% of the tweets are through suspected bots
A free video abstract to accompany this article can be found online at: https://www.youtube.com/watch?v= XGw3WMS5VbA © 2021 British Academy of Management andWiley Periodicals LLC. Published by JohnWiley & Sons Ltd, 9600 Gars- ington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA, 02148, USA.
Social Bots and the Spread of Disinformation in Social Media 1239
(Pew Research Center, 2018). These days, Twitter has become a vector for spreading misinformation (Oh, Agrawal and Rao, 2013).
The above indicates that a variety of AI agents are classified by unique functions (Russell and Norvig, 2016). The social actor, who enters the problem situation in interaction with others (net- worked actors) and events themselves, works to set goals and find procedures that can achieve the de- sired endpoint (Cantor and Kihlstrom, 2000). AI appears in different applications: information re- covery, text mining, expert systems, machine learn- ing, computational intelligence, computer vision, optimization, decision support and automation actions via robots or intelligent agents (Russell and Norvig, 2016). The latter is the focus of this study. These are the positive side of AI robots. However, we need to look at the other side as well. Social bots play vital roles in society; they can also be altered to perform a wide range of malicious activities targeting business enterprises, govern- ment agencies, non-governmental organizations (NGOs), political parties and SNSs. Information and data manipulation is an example of the dark side of social robots. For example, as Kudugunta and Ferrara (2018) mentioned during the 2010 US midterm elections, malicious social bots were em- ployed to support some candidates with fake news. Also, the debate on information manipulation has intensified today, with bots in social media and in- creased cyber attacks associated with the 2016 US presidential election. The allegations that Russia meddled in all big social media to harm the US election and increase political or social discord in the United States highlight that networked actors (humans, robots or machines) play important roles in framing situations for political gain.
Another example is Russian-linked groups, re- ported by the European Commission, that tried to undermine the credibility of the 2019 EU elections by disseminating false information through SNSs to influence votes (Satariano, 2019). Another study argues that some SNSs provide incentives for con- tent contribution (Tang, Gu andWhinston, 2012), persuading people to create content for peers. So- cial bots are also available to purchase.
There are firms active on the Net that sell fake followers to generate false popularity of a tweet ac- count. As noted by Yang et al. (2019), people can buy fake followers at a very low price. The major- ity of celebrities are among those who purchase fake followers on Twitter (Yang et al., 2019, p. 49).
Figure 1 shows a snapshot of a tweet in XML for- mat. Our dataset’s detected bot has tweeted 100 times, encouraging people to buy followers by of- fering them different sites to visit. The examples mentioned above reveal the chal-
lenging side of social bots through the spread of misinformation via social media, highlighting the dark side of recent technological advancements. It is important to note that humans are also behind many cases of spreading infodemic or fake news. To look at the challenging side of social bots, we
use actor-network theory (ANT) to support our argument, describing how malicious social bots manipulate social media (Shao et al., 2017) and influence their audience. As such, it is essential to investigate how social bots impact people by triggering actors’ actions in this context. We use ANT as a ‘toolbox to study meaning production, going from abstract structures – actants, to con- crete ones – actors (Latour, 1996, p. 373)’. In this context, ANT allows us to understand better the relationship between actors, meaning production, discourse or text (Latour, 1996) generated by the socially intelligent actors through the machine and deep learning approaches. In particular, we are in- terested in investigating the role of agents (humans and bots) in spreading false information about various socio-political and economic events to influence public opinion on the Twitter platform. Czarniawska (2006) noted that the notion of so-
cial has been extended from ‘humans only’ to ‘all actants that can be associated’ (p. 65). The lat- ter is the core concept of ANT, as articulated by Latour (2005). In this context, SNSs as a plat- form for social networking and knowledge shar- ing fit well within the framework of ANT. Couldry (2012) mentions that the new media whose history has been so important tomodernity’s shared world is a platform for transforming what was formerly called ‘mass’ media into the interface between person-to-person, person-to-object and object-to- object (e.g. bots retweet other bots).Moreover, this interaction within the context of an interweaved network of objects and humans makes ANT a useful tool to study social bots as entities aimed to generate content and ‘interact with humans to emulate and possibly alter their behaviour’ (Fer- rara et al., 2016, p. 96). Social bots are able to explore information to fill their profiles (Ferrara et al., 2016, p. 99). In this context, using the ANT, we examine the role of social bots through the fol- lowing research questions:
© 2021 British Academy of Management and Wiley Periodicals LLC.
1240 N. Hajli et al.
Figure 1. A hyperlink to buy followers [Colour figure can be viewed at wileyonlinelibrary.com]
RQ1: What is the role of social bots in shaping public opinion on SNSs? RQ2: What mechanism can be used to accu-
rately determine if the author of a given tweet is a bot or a human?
In the next section, we develop the paper by reviewing the related studies. In the third sec- tion, we discuss the research method, followed by data analysis using the machine learning approach to detect malicious social bots in the fourth sec- tion. Finally, we conclude with a discussion as well as limitations and recommendations for future work.
Review of related works
Bots or software robots appear in different re- alities and arrangements, including social media bots, chatbots and conversational AI. Bots play important roles in human life today. Different AI applications have been developed for industrial manufacturing, healthcare, transportation, avia- tion, financial institutes and governmental and public initiatives such as smart cities. In fact, the emergence of new technologies as enablers and multipliers is the backbone of today’s AI applica- tions, in that the Internet has made it possible to connect massively parallel AI systems to support these businesses. As mentioned earlier, malicious bots are explicitly designed with the purpose to harm.
Actor-network theory
In recent years, ANT has been applied in a wide range of scholarly research from information systems (IS) (Mwenya and Brown, 2017; Wal- sham, 1997) to healthcare (Iyamu and Mgudlwa, 2018; Lutz and Tamò, 2016), business (David and Halbert, 2014; Effah, 2012; Murdock and Varnes, 2018; Sarker et al., 2006) and education among others. Mwenya and Brown (2017) argue that a ‘Key tenet of ANT is generalized symmetry, which advocates that human and material actors
be viewed on the same analytic plane’ (p. 1). Their empirical study covering 36 recent scholarly publications in the IS domain shows that a wide range of IS studies have adopted ANT in several different ways. These studies highlight human and non-human actors’ vital role in information man- agement research. Iyamu and Mgudlwa (2018) reviewed big healthcare data from the lens of ANT. Using ANT, Lutz and Tamò (2016) investi- gate social and healthcare robots’ role as a threat to patients’ privacy within the healthcare systems. They emphasized that ANT is a descriptive, con- structivist approach that considers the agency of objects, concepts, ideas and the rationality of technology and society (p. 2).
Latour (1996) mentioned that earlier studies of social networks, no matter how interesting, con- cern themselves with the social relations of individ- ual human actors – their frequency, distribution, homogeneity and proximity. But to do so, it does not limit itself to the individual human actor; it ex- tends the word actor – or actant – to non-human, non-individual entities. It does not wish to add so- cial networks to social theory but to rebuild social theory out of networks (p. 369).
As effective multipurpose communication platforms, SNSs such as Twitter, Facebook, In- stagram, LinkedIn, Myspace and others play important roles in our daily lives. Christakis and Fowler (2009) view SNSs as ‘a kind of human superorganism. They grow and evolve. All sorts of things flow and move within them. This super- organism has its structure and function, and we became obsessed with understanding both. Seeing ourselves as part of a superorganism allows us to understand our actions, choices and experiences in a new light’ (p. xii). In this study, we view SNSs or social media in general as a collection of human, non-human objects as detailed by ANT. This view supports us in understanding the role of social bots in spreading misinformation, rumours and fake news within these sites and employing mecha- nisms to detect them promptly. This study debates that AI are integral parts of ANT. The integra- tion will help us understand better human and
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1241
non-human behaviour when they use technology- mediated social settings (Kling et al., 2003).
ANT also extends semiotics, which investi- gates how meaning is created and how meaning is communicated. ANT is a disparate family of material-semiotic tools, sensibilities and methods of analysis that treat everything in the social and natural worlds as a continuously generated effect of the webs of relations within which they are located. Nevertheless, actor networks do connect, and by connecting with one another, they provide an explanation of themselves, the only one there is for ANT (Latour, 1996). In this context, ANT extends the study of how signs and symbols (visual and linguistic) create meaning, but linking semi- otics to things instead of limiting them to meaning (Latour, 1996). Law (2008) argues that the ingredi- ents of ANT include ‘semiotic relationality (it is a network whose elements define and shape one an- other), heterogeneity (there are different kinds of actors, human and otherwise) and materiality not just the social’ (p. 146). In this context, an actor is always a network of elements that the social and technical elements are embedded in (Law, 2008). This means that it simply is not possible to explore the social without at the same time studying the ‘hows’ of relationships (p. 147); if all the world is relational, then so too are texts (Law, 2008). The actor network’s material semiotics explore the ‘hows’ to articulate new intellectual tools, sensibilities and questions (p. 148).
Semiotics is an integral part of the current study. As noted by Mattozzi (2019), an icon or likeness (i.e. a resemblance, as with a figurative image), an index (i.e. physical or causal relations), a symbol like a hashtag, an object or referent, a representation (i.e. the actual sign which repre- sents the object), provides a common ground for the exchange between ANT and semiotics (Law, 2008). In this context, semiotics as a ‘method’ allows describing the ‘interdefinition of actors and the chains of translations’ (Latour, 1998, p. 11). Or as precisely mentioned by Mol (2010, p. 257): ‘In semiotics, words do not point directly to a referent, but form part of a network of words. They acquire their meaning relationally, through their similarities with and differences from other words. In ANT this semiotic understanding of relatedness has been shifted on from language to the rest of reality. Thus, it is not simply the term, but the very phenomenon of its relations’ (cf. Mattozzi, 2019, p. 90). Finally, as mentioned
by Mattozzi, the main task of the empirical level of ANT is the material study of objects at the language level (Mattozzi, 2019).
AI-powered social bots
Digital technologies such as AI are progressively becoming key to reaching a competitive advantage in business. However, simultaneously, firms are threatened with a range of challenges of these technologies in the market. One of these chal- lenges is malicious social bots. These bots are a new form of bots that use social media to create content (Boshmaf et al., ; Lee, Eoff and Caverlee, 2011; Russell and Norvig, 2016; Varol et al., 2017; Woolley, 2016). However, Twitter is often used for research because it is open and easy to use (Ross et al., ). User-generated content on social me- dia influences organizations (Sheng et al., 2019), making social media an essential part of strate- gic management. As part of the realm of social intelligence (Russell and Norvig, 2016), social bots are important in this context. As Chu et al. (2012) and Ferrara et al. (2016) have mentioned, their proliferation has had both bad and good outcomes. In the case of the COVID-19 crisis, they can provide information useful for protecting society. Automated bots can also be useful for combining data from multiple sources for further analysis. However, malicious bots are sources of disinformation.1 For example, spammers and rogue agents (e.g. hackers) can manipulate bots to appeal to current profiles (Ferrara et al., 2016; Ghosh et al., 2012; Hu et al., 2013). Furthermore, hackers can use malicious bots to
produce more severe effects, such as generating anxiety and panic in emergencies like COVID-19 (Chakraborty et al., 2020; Shi et al., 2020), harm- ing a company’s status, swaying political opinions (Chu et al., 2012; Wang, 2010) and/or spread- ing rumours and fake news. According to MIT researchers, robots have successfully spread both true and false news faster in social media: in fact, the latter spread more easily because robots are more likely to be behind them (MIT Media Lab, 2018). Research shows that social bots play the spread of fake news online (Shao et al., 2017).
1According to the Merriam-Webster dictionary, disinfor- mation is defined as ‘false information deliberately and often covertly spread (as by the planting of rumors) in order to influence public opinion or obscure the truth’.
© 2021 British Academy of Management and Wiley Periodicals LLC.
1242 N. Hajli et al.
Moreover, ‘relatively few accounts are responsible for a large share of the traffic that carries misin- formation’ (p. 11). Therefore, this paper uncovers how offensive malicious social bots pose a threat by evaluating them using detection techniques and suggesting possible future study paths.
Our ANT approach
The four main components of ANT relevant to this study are actors, network interactions (e.g. so- cial media networks), meanings and action. The strength of ANT lies in its emphasis on humans and objects as participants in generating mean- ings (deliberation or manipulation) in the course of actions (Latour, 2005). ANT does not alter or transformmeanings; it aims to reveal themeanings without any alteration. As Latour (2005) empha- sized, the ‘black box’ notion of ANT is acting as an intermediary role in which it aims to transport meaning without transformation. The volume of information exchange that various agents generate is an interesting phenomenon of consideration, as they may directly or indirectly be associated with the characteristics of agents (e.g. agents’ risk aver- sion or risk-seeking behaviour, agents’ responsive behaviour, self-interest and so on).
We aim to analyse tweeter posts as they are without alteration, transformation, distortion or modified meaning. In addition, there are two broad categories of meanings (content) dissem- inated in social media by actors – deliberation (information) and manipulation (disinformation) – which in turn trigger actions. The latter is caused by bad actors (humans or malicious social bots) to distort communication discourse.
In the context of social media, the actors are integrated within the network platform. As such, ANT helps us separate AI-powered bots from the pool of existing actors active in social media such as Twitter. This allows us to investigate further the malicious activities of social bots in spreadingmis- information.
Broadly speaking, fake news has been studied based on multiple theoretical perspectives, includ- ing the style of the content (Zuckerman, DePaulo andRosenthal, 1981), distribution patterns and at- tributes of the user in creating fake news (Zhou and Zafarani, 2018). Some of the approaches to tackle these issues have been fact-checking with ex- perts, a machine learning algorithm, information comparisons, etc. Unique emotional patterns and
Figure 2. Social bot analytics processes
signals between fake news and real news are also studied, as manipulation requires emotional lan- guage cues (Ghanem, Rosso and Rangel, 2020).
Research method
Figure 2 outlines distinct phases of our approach to investigate our research questions.
As mentioned by Adams (2017), the rapidly advancing machine learning areas and AI con- tributed to powerful AI-enabled multi-modal social bots. The proposal developed below builds upon the multi-perspective framework for analysing agents’ behaviour in a networked environment, particularly Twitter, where we aim to generate a new understanding of social bots’ accelerated activities in spreading both real and false news through social media. The latter is of distinct interest to this study. The problem at hand strongly generalizes this setting. As Figure 2 shows, our goal is to formulate a methodological and technically supported model that includes text mining and machine learning-based classification of agents in Twitter and social bots’ role in this context.
As Zeng et al. (2010) have mentioned, social media analytics develops tools and techniques for collecting, analysing and visualizing social media data, motivated by the target application’s spe- cific requirements.We extend this definition by em- phasizing the importance of sharing the results with other stakeholders and the community at large.
Data
The PAN-20 evaluation lab provided data for this empirical study to detect fake news (PAN, 2020). Each set of data was shipped with the related la- belled meta-file as depicted in Figures 3 and 4. Our dataset was mixed with fake news generated by malicious social bots, humans and true stories. In addition, we had two sets of data in English and Spanish; we selected the English-based tweets.
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1243
Figure 3. XML format. Source: PAN (2020)
Figure 4. Meta-file in text format. Source: PAN (2020)
Data analysis: detecting AI-enabled social bots on Twitter
For detecting social bots and the spread of fake news, we received a dataset of 30,000 tweets in the English language from the PAN-20 project (PAN, 2020). These tweets were provided in the form of XML files. Each file represented one user and con- tained 100 tweets for that user. Separate files were provided that contained meta-data for the XML files.
The first challenge we faced was related to the XML format of shipped files (see Figure 3) not being suitable for natural language processing (NLP). As such, we processed data formatting to generate data frames2 suitable for our analysis. We split data randomly into two subsets comprising 80% of data or 24,000 tweets for training. For test- ing, we used the remaining 20% (6,000 tweets) to validate the dataset.
We performed three sets of algorithms in or- der to mine the tweet text, classify the content (RQ2) and finally analyse the classified data. For this section, we apply different machine learning algorithms to determine the best accuracy rates in response to our RQ2, as described below.
Text mining
We use text mining in this research. This method- ology discovers patterns and relationships in the
2The pandas library was used for generating DataFrame from XML files.
text (Batistič and van der Laken, 2019; Sheng et al., 2019; Thorpe et al., 2018). Mostafa (2013) argues that NLP applications made text mining possible. NLP is performed on text collections composed of tweets, known as a corpus. The corpus inher- its many hidden patterns in which further analysis is needed to confirm their relevance to decision- making factors. Text mining uses machine learning techniques to gain knowledge from a large dataset that can then be used in decision-making. Using SNSs data, textmining processes offer scholars un- derstanding about the opinions in the text collec- tion (Sharda, Delan and Turban, 2013).
Machine learning
Machine learning is one popular methodology for studying user-generated content (Larsen et al., 2019; Lukyanenko et al., 2017; Ptaszynski et al., 2019). Research shows that this method can sup- port companies to recognize applicable content generated by social media (Vermeer et al., 2019). In machine learning, information processors sort, assemble, simulate and classify information.
Classification
Classification is a process by which objects are di- vided into conceptually meaningful groups. The decision tree is among common classification tech- niques. Decision tree algorithms come from the family of supervised machine learning algorithms, where data is continuously split based on a pa- rameter. There are generally two types of decision trees: classification trees and regression trees. For our problem, we used classification trees. To solve our research problems outlined earlier
in RQ2, we used various text pre-processing tech- niques and applied multiple machine learning as well as deep learning algorithms. We fed Twitter data as input into our machine learning systems for file reading and processing to achieve accuracy, as described below. Figure 4 shows the meta-data file in the form of
a text file. As mentioned in the literature, fake news needs
to preserve stylistic features in the tweet to record information that can add value in evaluation. Hence, we used the pre-processing of content (Baziotis, Pelekis and Doulkeridis, 2017) in order to tokenize the content into unique tags before
© 2021 British Academy of Management and Wiley Periodicals LLC.
1244 N. Hajli et al.
passing it into ourmachine learningmodels. In this phase, we go through the following steps:
• Removing stop words using a natural language toolkit.
• Removing repeated words. • Converting emojis to text (e.g. a happy face emoji was converted to <HAPPY>).
• Removing punctuation marks. • Removing HTML tags. • Removing numbers, keeping only letters. • Replacing user mentions @username with
<USERMENTION>. • Performing stemming using a snowball stemmer. • Converting the content into a long string. • Lowercasing the text.
This is an important feature, because humans tend to have more diverse writing styles than bots, and bots tend to retweet more than humans. Despite the fact that both human and malicious social bots are behind the spread of fake news, there are differences between the two when it comes to the use of language and symbols. As mentioned by Varol et al. (2017), bots tend to retweet each other, meaning that simple bots frequently mention sophisticated bots and sophis- ticated bots retweet but do not mention humans due to their inability to get involved in expressive discourse with individuals. Also, individuals may retweet bots, but those humans do it by posting interesting content. As observed by Gilani et al. (2017), malicious social bots tend to share URLs and upload media content more frequently than humans, and humans usemore hashtags than bots. As such, indicators such as the average number of sentences, the average number of words, the average number of # symbols, @ symbols and the frequency of unique words are important indi- cators for distinguishing humans from malicious social bots (Gilani et al., 2017).
Results
After cleaning data for detecting fake news spread via malicious social bots, we deployed a text- analytic technique called an n-gram, composed of an n-character share of a longer text (Cavnar and Trenkle, 1994). To obtain the highest accu- racy rates (RQ2), we conducted six different ma- chine learning and classification approaches, along with two deep learning methods according to the
literature. The machine learning and classifica- tion approaches include multinomial naïve Bayes (Su, Shirab and Matwin, 2011), logistic regression (Dreiseitl andOhno-Machado, 2002), support vec- tor machine (Tong and Koller, 2001), decision tree (Kohavi andQuinlan, 2002), multilayer perceptron neural network (Pham et al., 2019) and K-nearest neighbour (Maillo et al., 2017). The deep learn- ing approach includes long short-term memory (LSTM) (Liu, Mi and Li, 2018) and bi-directional long short-termmemory (Bi-LSTM) with the self- attention option (Zhou et al., 2016).
In standard machine learning, multinomial naïve Bayes returned the best result in its category, while Bi-LSTM (with self-attention) returned the best deep learning results. In the following sections, we explain the deep learning approaches.
Deep learning approaches. It is possible to gen- erate output that mimics human behaviour, or the so-called AI-powered (enabled) social bots with companion generative networks (Foysal, Islam and Rahaman, 2019). LSTM is an artificial recurrent neural networks (RNNs) architecture classified as a deep feedforward neural network for classification of sound and signal categories (Le et al., 2019; Sak, Senior and Beaufays, 2014), handwriting recognition (Graves et al., 2009), speech tagging (Huang, Xu and Yu, 2015) and key phrase extraction as part of NLP (Alzaidy, Caragea and Giles, 2019). LSTM has a memory structure with self-connection links that store the network’s temporal state at each time step. The Bi-LSTM networks operate on the input sequence in forward and backward directions to decide the local context. The attention-based Bi-LSTM is designed to collect the most important semantic information in a sentence (Zhou et al., 2016).
Measuring accuracy rates. Accuracy is a measure of performance and is calculated as the percent- age of correct predictions (ratio of correct predic- tions to number of total predictions). As depicted in Figure 5, a single case problem has two classifi- cations – positive and negative – in which the true positive (TP) and true negative (TN) are desired.
However, a false positive (FP) happens when the outcome is incorrectly predicted as positive when it is actually negative. A false negative (FN) occurs when the outcome is incorrectly predicted as negative when it is actually positive (Lin and Chen, 2009; Schwenke and Schering, 2007; Yin et al., 2017). As a measure of performance, Eq. (1)
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1245
Table 1. Accuracy rates for different classification methods
Model Accuracy (validation set)
Multilayer perceptron neural network 68 SVM 72.2 LSTM 74.2 KNN 74.7 Decision tree 76.7 Logistic regression 78.2 Multinominal naïve Bayes 79.1 Bi-LSTM with self-attention 79.7
SVM = support vector machine; LSTM = long short-term memory; KNN = k-nearest neighbour; Bi-LSTM = bi-directional long short-term memory.
Figure 5. A single classification model
calculates the accuracy rate as the proportion of correctly predicted (TP and TN) to the total value of all predictions:
Accuracy = (TP + TN) (TP + TN + FP + FN)
(1)
Findings and discussion
The above-mentioned algorithms were run on the training and test sets and achieved accuracies for the tasks as reported in Table 1. As indicated, the machine learning methods using popular classi- fiers, including logistic regression and multinomial naïve Bayes, still perform well for the text mining studies described.However, using a propermethod such as Bi-LSTMwith deepmachine learning with self-attention option offers the best possible results within a reasonable time frame. As mentioned by Huang, Xu and Yu (2015), within the context of bi-directional LSTM architecture, data in this sys- tem uses back-propagation through time (BPTT), which leads to improved organization, search, re- trieval and recommendation of various documents (Alzaidy, Caragea and Giles, 2019).
Our attention-based Bi-LSTM model was trained with eight epochs; the hidden layers were
set to size 50, dropout was set to 0.2 and ‘Adagrad- Trainer’was selected. The Python AdagradTrainer function is essentially an optimizer that performs the stochastic gradient descent procedure for deep neural networks. The empirical studies show that the deep
learning Bi-LSTM model outperforms different algorithms designed for text classification and analysis, in particular when it comes to larger datasets (AlKhwiter and Al-Twairesh, 2020; Jang et al., 2020; Minaee, Azimi and Abdolrashidi, 2019; Shaid, Zammer and Muneeb, 2020). As indicated by ANT, social bots and humans
are integrated with a web of relationships. Each actor plays an important role in farming situa- tions to impact public opinion and policy-making processes regarding social, economic or political matters. However, as mentioned earlier, many bots are created solely to spread misinformation, ru- mours and spams. One of the key aspects used as an indicator is social context (Zhou and Zafarani, 2018) and the distribution pattern of real and fake news. We are using a text segmentation algorithm such as n-gram to effectively classify the text writ- ing styles for the classification of social bots and humans (RQ2). Our findings answer our research question RQ2 in identifying fake news written by humans versus bots with an accuracy rate of 79.7% due to the use of a more efficient predictive anal- ysis algorithm. This forms one of our key findings that the algorithm could filter out almost 80% of malicious social bot-generated tweets intended to harm public trust and/or spread misinformation. To highlight this, Figure 6 shows a snapshot
of a malicious social bot that we detected in our dataset. It is important to note that the timing of our collected dataset was linked to the 2020 US presidential election; as such, many of the tweet
© 2021 British Academy of Management and Wiley Periodicals LLC.
1246 N. Hajli et al.
Figure 6. A malicious social bot [Colour figure can be viewed at wileyonlinelibrary.com]
messages were related to this event. The figure shows an example of these messages.
We tracedTrue Trumpers in Twitter. As shown in Figure 7, the account was created in August 2017, with no followers but 4,423 tweets. It does not have a human profile picture either.
Figure 8 shows completely false stories tweeted as collections by True Trumpers. In fact, this ma- licious social bot operated from an Eastern Euro- pean country, according to Buzzfeed (2017).
While much research emphasizes the value of social media for people and organizations (Candi et al., 2018; Leonardi, 2017; Mangold and Faulds, 2009; Sigfusson and Chetty, 2013; Williams, Du and Zhang, 2020), other research highlights un- avoidably malicious counterparts to the benefits of social media that bring about harmful con- sequences in community discourse (Miranda, Young and Yetgin, 2016). Emerging technologies such as AI empower bots on social media plat- forms, causing them to develop into social bots. The proliferation of bots has both bad and good outcomes. Our research provides an algorithm to differentiate tweets written by a human from those by bots with an accuracy rate of 79.7% due to the use of a more efficient predictive analysis algorithm. Public and private organizations, as well as individuals, can apply this algorithm to filter out almost 80% of tweets generated by social bots aimed to harm public trust and/or spread dis- information otherwise harmful to public opinion about a brand, political organizations, political actors, celebrities or commercial enterprises.
As mentioned by Latour (1996), we used ANT as a ‘toolbox’ to investigate our research ques- tion RQ1 in order ‘to study meaning production… going from abstract structure – actants, to con- crete ones – actors, in which an actant can be anything provided it is granted to be the action source’ (Latour, 1996, p. 373) (see Figure 1). We deployed ANT to critically investigate the fact that SNSs and social media are not just ‘a kind of human superorganism’ (Christakis and Fowler, 2009), and/or the networking aspects of SNSs ‘emphasize relationship initiation, often between
strangers’ (Boyd and Ellison, 2007, p. 211). In re- sponse to RQ2, we empirically investigated the role of social bots in SNSs. We showed that the bots exist, but they are also primary sources of dissemi- nating fake news, rumours and disinformation.We also empirically tested a range of popular algo- rithms using 30,000 tweets to determine the most effective algorithm that provides us with the mech- anism to detect the social bots (RQ2) with a high accuracy rate.
Within the context of ANT, automated bots have more power in the network as they are active 24/7 a week for tweeting and retweeting subjects that they are programmed to do. They can be the source of disinformation. Research shows (Ross et al., ) that when social bots encourage a definite view on social media, they might create a situation that gives the false impression that the ‘bot opin- ion’ is shared by more individuals than in reality. Accordingly, individuals who agree with that in- formation get confidence to express the matter in interactions with other network peers, whereas in- dividuals who disagree keep quiet for fear of being socially isolated (Ross et al., 2019). The bottom line is the fact that social bots are one of the emerg- ing issues in management. They are widely used to blackmail a wide range of objects and actors.
Theoretical and practical implications
Theoretical implications. Plenty of studies have been undertaken to investigate the use of social bots in different fields (e.g. Abokhodair, Yoo and McDonald, 2015; Forelle et al., 2015). This re- search examines towhat extent social bots affect an individual’s opinion formation. It also contributes to the growing research in AI, such as social bots – particularly how social bots spread a given on- line opinion over social media platforms, leading to the misconception that other humans share the ‘bot opinion’. It explains an admissible mechanism of opinion manipulation based on the theory of opinion formation. Numerous studies have inves- tigated different aspects of social bots, described different techniques to recognize social bots (e.g.
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1247
F ig ur e 7. A ty pi ca ls oc ia lb ot ac co un t [C
ol ou r fig ur e ca n be vi ew ed at w ile yo nl in el ib ra ry .c om
]
© 2021 British Academy of Management and Wiley Periodicals LLC.
1248 N. Hajli et al.
Figure 8. A sample of posts by True Trumpers. Source: Buzzfeed
Subrahmanian et al., 2016) or explained their be- haviour (e.g. Hegelich and Janetzko, 2016).
This research integrates AI and ANT to under- stand better human and non-human behaviour in the use of social media as a platform for com- munication discourse. Recent studies related to social bots in social media were mainly focused on technological and software engineering aspects (Jang et al., 2020; Minaee, Azimi and Abdol- rashidi, 2019; Ross et al., ; Shaid, Zammer and Muneeb, 2020; Wang, 2010; Zhou et al., 2016). Our research is among the few to utilize a theory- based focus to look at the role of malicious so- cial bots and the spread of disinformation in social media through experimental research. Therefore, this research, with an interdisciplinary approach, borrows ANT from information systems to inves- tigate the role of AI in management.
As Walsham (1997) mentioned, ANT looks at the actors who are linked by associations of heterogeneous networks of aligned interests. A vital feature of ANT is that performers include human and non-human actors such as techno- logical objects (Walsham, 1997). They are part of hybrid networks such as SNSs. Besides, ANT
is both a ‘theory and methodology combined’ (Walsham, 1997, p. 469). ANT not only provides theoretical concepts to view the fundamentals of the world, but also suggests that it is precisely these fundamentals that need to be outlined in empirical work (Walsham, 1997). This study has this as its main focus, and it is ‘of course, no small task for a complex network’ (Walsham, 1997, p. 470) – such as social media networks.
Practical implications. As mentioned above, ma- licious social bots have generated millions of so- cial media pages containing false, unreliable and misleading information aimed at harming brands. Early detection of these malicious bots, some of which can change their behaviour and charac- teristics, is an important task for trust-building, whether related to a business or a particular brand and/or socio-political matters. From a managerial perspective, users, social actors and organizations can use our deep learning tool for early detection of harmful social bots before they can spread mis- information on social media. We obtained almost 80% accuracy in our deep learning classification approach (human or bot) using an attention-based
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1249
Bi-LSTM. Therefore, this tool is useful for manag- ing information and disinformation on SNSs.
Besides, the results of our study highlight the fact that extra security measures – such as two- factor identification (e.g. a combination of pass- word and anti-bot measures such as smart captcha or token) – should be in place to limit or block unwanted bots from spreading disinformation in SNSs, including Twitter.
Conclusion, limitations and future work
This study extends the ANT to the deep learning model of text mining from the perspective of the semiotics paradigm. We have also contributed to ANT by demonstrating the interrelationship be- tween actors (humans, bots) and a web of words, symbols, signs, tokens, icons, etc., to actors on the ANT network. As noted byMattozzi (2019), ANT offers a ‘methodological middle ground in between the theoretical-conceptual and the empirical ones’ (p. 97), as provided by this study. In addition, as Mattozzi highlighted, the main task of the empir- ical level of ANT is the material study of objects at the language level (Mattozzi, 2019). In this con- text, RQ1 and RQ2 integrate to understand bet- ter the content of tweets generated by actors and, in particular, the malicious bots in shaping public opinion on SNSs.
Our experimental research is set up to examine the role of AI and social bots in spreading dis- information. Social bots can harm a company’s status by spreading rumours and fake news about a particular brand. Social bots have successfully accelerated the spread of both true and false news in social media; in fact, the latter spread more because robots are more likely behind it. These days, thoughts expressed in SNSs play a key role in influencing actors across all domains, including business. Therefore, this paper reveals the probable threats of offensive social bots by evaluating them using detection techniques. Overall, the findings indicate the theoretical potential for using auto- mated robot accounts to form an online opinion. The results show that willingness to express an opinion and form an online opinion is affected by social bots on SNSs. It has been discussed that although bots are considered less credible than humans, they still have a significant impact on on- line public opinion. This research emphasizes that the propagation of disinformation increases social
bots’ activity, aimed at spreading unverified infor- mation. Drawing from ANT, this research exam- ines human and non-human actors’ roles in SNSs, particularly Twitter, to understand better social bots’ role in spreading spam and false information. This research has limitations, like other studies.
The lack of meta-data is one of them. As depicted in Figure 4, the meta-data was masked out and re- placed with a code to anonymize the tweet author’s actual name. The meta-data contains valuable in- formation about the author’s information, like the author’s name, profile picture, ID number, physical location, followers and the time stamp of posts or retweets, among others. If the meta-data and the content feed into our deep learning machine, the accuracy will be increased drastically due to more information provided to the system. Particularly, meta-data provides valuable information for the purpose of author classification. As mentioned by Stieglitz et al. (2017a, 2017b), many bots lack basic account information like name or profile pictures. While regular users get access from front-end web- sites, bots obtain access through a site’s application programming interface (API). As the API of Twit- ter is especially accessible, many social bots focus on this platform (Stieglitz et al., 2017a) because it offers a wide range of resources and a faster com- munication platform (Crains and Shetty, 2020). We used only English-language tweets, and this
is another limitation of our research. As men- tioned, we had two sets of Twitter data, in English and Spanish; we had the option to select one of the possible languages or both. We selected only the English language. Our focus in this study was text-based communication, while ourmodel is also designed to perform well with non-text communi- cation such as voice (speech), image and video clas- sification, as the deep learning Bi-LSTMmethod is chartered to do so. In the future, new research may consider embedding TF-IDF weighting (Rangel and Rosso, 2019) and studying document embed- ding (Daelemans et al., 2019). Using syntactic n- grams is another option for future research direc- tion (Potthast et al., 2019).
References
Abokhodair, N., D. Yoo and D. W. McDonald (2015). ‘Dissect- ing a social botnet’. In Proceedings of the 18th ACM Confer- ence on Computer Supported Cooperative Work & Social Com- puting – CSCW ’15, Vancouver, BC, Canada, pp. 839–851. New York: ACM.
Adams, T. (2017). ‘AI-powered social bots’, arXiv:1706.05143.
© 2021 British Academy of Management and Wiley Periodicals LLC.
1250 N. Hajli et al.
AlKhwiter, W. and N. Al-Twairesh (2020). ‘Part-of-speech tag- ging for Arabic tweets using CRF and Bi-LSTM’, Computer Speech & Language, 65, art. 101138.
Alzaidy, R., C. Caragea and C. L. Giles (2019). ‘Bi-LSTM-CRF sequence labeling for keyphrase extraction from scholarly doc- uments’. In The World Wide Web Conference, pp. 2551–2557.
Batistič, S. and P. van der Laken (2019). ‘History, evolution and future of big data and analytics: a bibliometric analysis of its relationship to performance in organisations’, British Journal of Management, 30, pp. 229–251.
Baziotis, C., N. Pelekis and C. Doulkeridis (2017). ‘Datasto- ries at Semeval-2017 Task 4: deep LSTM with attention for message-level and topic-based sentiment analysis’. In Proceed- ings of the 11th InternationalWorkshop on Semantic Evaluation (SemEval-2017), pp. 747–754.
Boyd, D. M. and N. B. Ellison (2007). ‘Social network sites: definition, history, and scholarship’, Journal of Computer- Mediated Communication, 13, pp. 210–230.
Buzzfeed (2017). ‘This pro-Trump website run from Eastern Europe may be the worst thing on the Internet’. Avail- able at https://www.buzzfeednews.com/article/craigsilverman/ anti-muslim-traffic-arbitrage-is-a-thing. Accessed May 12, 2020.
Candi, M., D. L. Roberts, T.Marion andG. Barczak (2018). ‘So- cial strategy to gain knowledge for innovation’, British Journal of Management, 29, pp. 731–749.
Cantor, N. and J. F. Kihlstrom (2000). ‘Social intelligence’. In R. J. Sternberg (ed.), Handbook of Intelligence, vol. 2, pp. 359– 379. Cambridge: Cambridge University Press.
Cavnar, W. B. and J. M. Trenkle (1994). ‘N-gram-based text cat- egorisation’. In Proceedings of SDAIR-94, 3rd Annual Sympo- sium onDocument Analysis and Information Retrieval, pp. 161– 175.
Chakraborty, K., S. Bhatia, S. Bhattacharyya, J. Platos, R. Bag and A. E. Hassanien (2020). ‘Sentiment analysis of COVID- 19 tweets by deep learning classifiers—a study to show how popularity is affecting accuracy in social media’, Applied Soft Computing, 97, art. 106754.
Christakis, N. A. and J. H. Fowler (2009). Connected: The Sur- prising Power of Our Social Networks andHowThey ShapeOur Lives. London: Little, Brown Spark.
Chu, Z., S. Gianvecchio, H.Wang and S. Jajodia (2012). ‘Detect- ing automation of Twitter accounts: are you a human, bot, or cyborg?’, IEEE Transactions on Dependable and Secure Com- puting, 9, pp. 811–824.
Couldry, N. (2012). Media, Society, World: Social Theory and Digital Media Practice. Cambridge: Polity Press.
Crains, I. and P. Shetty (2020). ‘Introducing a new and improved Twitter API’. Available at https://blog.twitter.com/developer/ en_us/topics/tools/2020/introducing_new_twitter_api.html. Accessed April 11, 2021.
Czarniawska, B. (2006). ‘Bruno Latour: reassembling the social: an introduction to actor-network theory’, Organization Stud- ies, 27, pp. 1553–1557.
Daelemans, W., M. Kestemont, E. Manjavacas, M. Potthast, F. Rangel, P. Rosso et al. (2019). ‘Overview of PAN 2019: bots and gender profiling, celebrity profiling, cross-domain author- ship attribution and style change detection’. In International Conference of the Cross-Language Evaluation Forum for Euro- pean Languages, pp. 402–416. Cham: Springer.
David, L. and L. Halbert (2014). ‘Finance capital, actor-network theory and the struggle over calculative agencies in the busi-
ness property markets of Mexico City Metropolitan Region’, Regional Studies, 48, 516–529.
De Lima Salge, C. A. and N. Berente (2017). ‘Is that social bot behaving unethically?’, Communications of the ACM, 60, pp. 29–31.
Dreiseitl, S. and L. Ohno-Machado (2002). ‘Logistic regression and artificial neural network classification models: a method- ology review’, Journal of Biomedical Informatics, 35, pp. 352– 359.
Effah, J. (2012). ‘Mobilizing culture for e-business in developing countries: an Actor network theory account’, The Electronic Journal of Information Systems in Developing Countries, 52, pp. 1–17.
Ferrara, E., O. Varol, C. Davis, F. Menczer and A. Flammini (2016). ‘The rise of social bots’, Communications of the ACM, 59, pp. 96–104.
Forelle, M. C., P. N. Howard, A.Monroy-Hernandez and S. Sav- age (2015). ‘Political bots and themanipulation of public opin- ion in Venezuela’, SSRN Electronic Journal, 14, pp. 57–74.
Foysal, A., S. Islam and T. Rahaman (2019). ‘Classification of AI powered social bots on Twitter by sentiment analysis and data mining through SVM’, International Journal of Computer Applications, 117, pp. 13–19.
Ghanem, B., P. Rosso and F. Rangel (2020). ‘An emotional anal- ysis of false information in social media and news articles’, ACM Transactions on Internet Technology (TOIT), 20, pp. 1– 18.
Ghosh, S., B. Viswanath, F. Kooti, N. K. Sharma, G. Korlam, F. Benevenuto et al. (2012). ‘Understanding and combating link farming in the Twitter social network’. In Proceedings of the 21st International Conference on the World Wide Web.
Gilani, Z., R. Farahbakhsh, G. Tyson, L. Wang and J. Crowcroft (2017). ‘Of bots and humans (on Twitter)’. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 349–354.
Graves, A., M. Liwicki, S. Fernandez, R. Bertolami, H. Bunke and J. Schmidhuber (2009). ‘A novel connectionist system for improved unconstrained handwriting recognition’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, pp. 855–868.
Hegelich, S. and D. Janetzko (2016). ‘Are social bots on Twitter political actors? Empirical evidence from a Ukrainian social botnet’. In Proceedings of the Tenth International Conference on Weblogs and Social Media (ICWSM-2016), Cologne, Ger- many, pp. 579–582. Palo Alto, CA: AAAI Press.
Hu, X., J. Tang, Y. Zhang andH. Liu (2013). ‘Social spammer de- tection in microblogging’. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence.
Huang, M.-H. and R. T. Rust (2020). ‘Engaged to a robot? The role of AI in service’, Journal of Service Research, https://doi. org/10.1094/670520902266.
Huang, Z., W. Xu and K. Yu (2015). ‘Bidirectional LSTM-CRF models for sequence tagging’, arXiv:1508.01991.
Iyamu, T. and S.Mgudlwa (2018). ‘Transformation of healthcare big data through the lens of actor network theory’, Interna- tional Journal of Healthcare Management, 11, pp. 182–192.
Jang, B., M. Kim, G. Harerimana, S. U. Kang and J. W. Kim (2020). ‘Bi-LSTM model to increase accuracy in text classifi- cation: combiningWord2vec CNN and attention mechanism’, Applied Sciences, 10, p. 5841.
Kling, R., G. McKim and A. King (2003). ‘A bit more to IT: Scholarly communication forums as socio-technical
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1251
interaction networks’, Journal of the American Society for In- formation Science and Technology, 54, pp. 46–67.
Kohavi, R. and J. R. Quinlan (2002). ‘Data mining tasks and methods: classification: decision-tree discovery’. In W. Klös- gen and J. M. Zytkow (eds), Handbook of Data Mining and Knowledge Discovery, pp. 267–276. New York: Oxford Univer- sity Press.
Kudugunta, S. and E. Ferrara (2018). ‘Deep neural networks for bot detection’, Information Sciences, 467, pp. 312–322.
Larsen,K.R.,D.Hovorka,A.Dennis and J.D.West (2019). ‘Un- derstanding the elephant: the discourse approach to boundary identification and corpus construction for theory review arti- cles’, Journal of the Association for Information Systems, 20, p. 15.
Latour, B. (1996). ‘On actor-network theory: a few clarifications’, Soziale Welt, 47, pp. 369–381.
Latour, B. (1998). ‘A relativistic account of Einstein’s relativity’, Social Studies of Science, 18, pp. 3–44.
Latour, B. (2005). Reassembling the Social: An Introduction to Actor-Network Theory. Oxford: Oxford University Press.
Law, J. (2008). ‘Actor network theory and material semiotics’. In B.S. Turner (ed.), The New Blackwell Companion to Social Theory, pp. 141–158. Oxford: Wiley-Blackwell.
Le, T., M. T. Vo, B. Vo, E. Hwang, S. Rho and S. W. Baik (2019). ‘Improving electric energy consumption prediction us- ing CNN and Bi-LSTM’, Applied Sciences, 9, p. 4237.
Lee, K., B. D. Eoff and J. Caverlee (2011). ‘Seven months with the devils: a long-term study of content polluters on Twitter’. In Fifth International AAAI Conference on Weblogs and Social Media.
Leonardi, P.M. (2017). ‘The social media revolution: sharing and learning in the age of leaky knowledge’, Information and Or- ganization, 27, pp. 47–59.
Lin, S. W. and S. C. Chen (2009). ‘PSOLDA: a particle swarm optimization approach for enhancing classification accuracy rate of linear discriminant analysis’, Applied Soft Computing, 9, pp. 1008–1015.
Liu, H., X. Mi and Y. Li (2018). ‘Smart multi-step deep learning model for wind speed forecasting based on variational mode decomposition, singular spectrum analysis, LSTM network and ELM’, Energy Conversion and Management, 159, pp. 54– 64.
Lukyanenko,R., J. Parsons, Y.Wiersma,G.Wachinger, B.Huber and R. Meldt (2017). ‘Representing crowd knowledge: guide- lines for conceptualmodeling of user-generated content’, Jour- nal of the Association for Information Systems, 18, p. 2.
Lutz, C. and A. Tamò (2016). Privacy and healthcare robots–an ant analysis. We Robot.
Maillo, J., S. Ramírez, I. Triguero and F. Herrera (2017). ‘kNN- IS: an iterative spark-based design of the k-nearest neighbors classifier for big data’, Knowledge-Based Systems, 117, pp. 3– 15.
Mangold, W. G. and D. J. Faulds (2009). ‘Social media: the new hybrid element of the promotion mix’, Business Horizons, 52, pp. 357–365.
Mattozzi, A. (2019). ‘What can ANT still learn from semiotics’. In A. Blok, I. Farias and C. Roberts (eds),The Routledge Com- panion to Actor-Network Theory, pp. 87–100. Abingdon: Rout- ledge.
Minaee, S., E. Azimi and A. Abdolrashidi (2019). ‘Deep- sentiment: sentiment analysis using ensemble of CNN and Bi- LSTM models’, arXiv:1904.04206.
Miranda, S. M., A. Young and E. Yetgin (2016). ‘Are social me- dia emancipatory or hegemonic? Societal effects of mass me- dia digitisation in the case of the SOPA discourse’,MIS Quar- terly, 40, pp. 303–329.
MIT Media Lab (2018). ‘The spread of true and false informa- tion online’. Available at https://www.media.mit.edu/projects/ the-spread-of-false-and-true-info-online/overview/. Accessed June 11, 2019.
Mol, A. (2010). ‘Actor-network theory: sensitive terms and enduring tensions’, Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 50, pp. 253–269.
Moon, S. andW.A.Kamakura (2017). ‘A picture is worth a thou- sand words: translating product reviews into a product posi- tioning map’, International Journal of Research in Marketing, 34, pp. 265–285.
Mostafa, M. M. (2013). ‘More than words: social networks’ text mining for consumer brand sentiments’, Expert Systems with Applications, 40, pp. 4241–4251.
Murdock, K. A. and C. J. Varnes (2018). ‘Beyond effectua- tion: analysing the transformation of business ideas into ven- tures using actor-network theory’, International Journal of En- trepreneurial Behavior & Research.
Mwenya, J. K. and I. Brown (2017). ‘Actor-network theory in IS research: critique on application of the principle of general- ized symmetry’. In Proceedings of the South African Institute of Computer Scientists and Information Technologists, pp. 1– 10.
Nadeem, W., M. Juntunen, N. Hajli and M. Tajvidi (2019). ‘The role of ethical perceptions in consumers’ participation and value co-creation on sharing economy platforms’, Journal of Business Ethics, 169, pp. 421–441.
Oh, O., M. Agrawal and H. R. Rao (2013). ‘Community intelli- gence and social media services: a rumor theoretic analysis of tweets during social crises’,MIS Quarterly, 37, pp. 407–426.
PAN (2020). ‘Cross-domain authorship attribution 2019’. Available at https://pan.webis.de/clef20/pan20-web/author- profiling.html. Accessed June 11, 2019.
Pew Research Center (2018). ‘Q&A: How Pew Research Cen- ter identified bots on Twitter’. Available at https://www. pewresearch.org/fact-tank/2018/04/19/qa-how-pew-research- center-identified-bots-on-twitter. Accessed November 20, 2020.
Pham, B. T., M. D. Nguyen, K. T. T. Bui, I. Prakash, K. Chapi and D. T. Bui (2019). ‘A novel artificial intelligence ap- proach based on multi-layer perceptron neural network and biogeography-based optimisation for predicting coefficient of consolidation of soil’, Catena, 173, pp. 302–311.
Potthast, M., T. Gollub, M. Wiegmann and B. Stein (2019). ‘TIRA integrated research architecture’. In N. Ferro and C. Peters (eds), Information Retrieval Evaluation in a Changing World – Lessons Learned from 20 Years of CLEF. New York: Springer.
Ptaszynski, M., P. Lempa, F. Masui, Y. Kimura, R. Rzepka, K. Araki et al. (2019). ‘Brute-force sentence pattern extortion from harmfulmessages for cyberbullying detection’, Journal of the Association for Information Systems, 20, p. 4.
Rangel, F. and P. Rosso (2019). ‘Overview of the 7th author pro- filing task at PAN 2019: bots and gender profiling’. In L. Cap- pellato, N. Ferro, H. Müller and D. Losada (eds), CLEF 2019 Labs and Workshops, Notebook Papers.
Russell, S. and P. Norvig (2016).Artificial Intelligence: AModern Approach, 3rd edn. Harlow: Pearson Education.
© 2021 British Academy of Management and Wiley Periodicals LLC.
1252 N. Hajli et al.
Sak, H., A. W. Senior and F. Beaufays (2014). ‘Long short-term memory recurrent neural network architectures for large scale acoustic modeling’.
Sarker, S., S. Sarker and A. Sidorova (2006). ‘Understanding business process change failure: an actor-network perspective’, Journal of Management Information Systems, 23, pp. 51–86.
Satariano, A. (2019). ‘Russia sought to use social media to influence E.U. vote, report finds’. Avail- able at https://www.nytimes.com/2019/06/14/business/ eu-elections-russia-misinformation.html. Accessed January 15, 2020.
Schwenke, C. and A. G. Schering (2007). ‘True positives, true negatives, false positives, false negatives’. InWiley Encyclope- dia of Clinical Trials. Chichester: Wiley.
Shahid, F., A. Zameer and M. Muneeb (2020). ‘Predictions for COVID-19 with deep learning models of LSTM, GRU, and Bi-LSTM’, Chaos, Solitons & Fractals, 140, art. 110212.
Shao, C., G. L. Ciampaglia, O. Varol, A. Flammini and F. Menczer (2017). ‘The spread of fake news by social bots’, arXiv:1707.07592.
Sharda, R., D. Delan and E. Turban (2013). Business Intelli- gence, Analytics, and Data Science: A Managerial Perspective, 4th edn. New York: Pearson.
Sheng, J., J. Amankwah-Amoah, X. Wang and Z. Khan (2019). ‘Managerial responses to online reviews: a text analytics ap- proach’, British Journal of Management, 30, pp. 315–327.
Shi, P., Z. Zhang andK.K. R. Choo (2019). ‘Detectingmalicious social bots based on clickstream sequences’, IEEE Access, 7, pp. 28855–28862.
Shi, W., D. Liu, J. Yang, J. Zhang, S. Wen and J. Su (2020). ‘Social bots’ sentiment engagement in health emergencies: a topic-based analysis of the COVID-19 pandemic discussions on Twitter’, International Journal of Environmental Research and Public Health, 17, p. 8701.
Sigfusson, T. and S. Chetty (2013). ‘Building international en- trepreneurial virtual networks in cyberspace’, Journal of World Business, 48, pp. 260–270.
Stieglitz, S., F. Brachten, D. Berthelé, M. Schlaus, C. Vene- topoulou and D. Veutgen (2017a). ‘Do social bots (still) act different to humans? Comparing metrics of social bots with those of humans’. In International Conference on So- cial Computing and Social Media, pp. 379–395. Cham: Springer.
Stieglitz, S., F. Brachten, B. Ross and A.-K. Jung (2017b). ‘Do social bots dream of electric sheep? A categorisation of social media bot accounts’. In Proceedings of the Australasian Con- ference on Information Systems, Hobart, Tasmania.
Su, J., J. S. Shirab and S. Matwin (2011). ‘Large scale text clas- sification using semi-supervised multinomial naive Bayes’. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 97–104.
Subrahmanian, V. S., A. Azaria, S. Durst, V.Kagan, A.Galstyan, K. Lerman et al. (2016). ‘The DARPA Twitter bot challenge’, Computer, 49, pp. 38–46.
Tang, Q., B. Gu and A. B. Whinston (2012). ‘Content contribu- tion for revenue sharing and reputation in social media: a dy- namic structural model’, Journal of Management Information Systems, 29, pp. 41–76.
Thorpe, A., R. Craig, D. Tourish, G. Hadikin and S. Batistic (2018). ‘“Environment” submissions in the UK’s Research Ex- cellence Framework 2014’, British Journal of Management, 29, pp. 571–587.
Tong, S. and D. Koller (2001). ‘Support vector machine active learning with applications to text classification’, Journal of Machine Learning Research, 2, pp. 45–66.
Varol, O., E. Ferrara, C. A. Davis, F. Menczer and A. Flammini (2017). ‘Online human–bot interactions: detection, estimation, and characterisation’. In Proceedings of the Eleventh Interna- tional Conference on Web and Social Media (ICWSM-2017), Montréal, Québec, Canada, pp. 280–289. PaloAlto, CA:AAAI Press.
Vermeer, S. A. M., T. Araujo, S. F. Bernritter and G. van Noort (2019). ‘Seeing the wood for the trees: how machine learn- ing can help firms in identifying relevant electronic word-of- mouth in social media’, International Journal of Research in Marketing, 36, pp. 492–508.
Walsham,G. (1997). ‘Actor-network theory and IS research: cur- rent status and future prospects’. In A. S. Lee, J. Liebenau and J. I. DeGross (eds), Information Systems and Qualitative Re- search, pp. 466–480. Boston, MA: Springer.
Wang, A. H. (2010). ‘Detecting spam bots in online social net- working websites: a machine learning approach’. In 24th An- nual IFIP WG 11.3 Working Conference on Data and Applica- tions Security.
Wang, X., M. Tajvidi, X. Lin and N. Hajli (2019). ‘Towards an ethical and trustworthy social commerce community for brand value co-creation: a trust–commitment perspective’, Journal of Business Ethics, 167, pp. 137–152.
Williams, C., J. Du and H. Zhang (2020). ‘International orienta- tion of Chinese internet SMEs: direct and indirect effects of foreign and indigenous social networking site use’, Journal of World Business, 55, art. 101051.
Woolley, S. C. (2016). ‘Automating power: social bot interference in global politics’, First Monday, 21, https://doi.org/10.5210/ fm.v21i4.6161. Accessed December 8, 2020.
Yang, K. C., O. Varol, C. A. Davis, E. Ferrara, A. Flammini and F. Menczer (2019). ‘Arming the public with artificial intelli- gence to counter social bots’, Human Behavior and Emerging Technologies, 1, pp. 48–61.
Yin, C., Y. Zhu, J. Fei and X. He (2017). ‘A deep learning approach for intrusion detection using recurrent neural net- works’, IEEE Access, 5, pp. 21954–21961.
Zeng, D., H. Chen, R. Lusch and S. H. Li (2010). ‘Social media analytics and intelligence’, IEEE Intelligent Systems, 25, pp. 13–16.
Zhou, P., W. Shi, J. Tian, Z. Qi, B. Li, H. Hao and B. Xu (2016). ‘Attention-based bidirectional long short-term memory net- works for relation classification’. InProceedings of the 54thAn- nual Meeting of the Association for Computational Linguistics, vol. 2, pp. 207–212.
Zhou, X. and R. Zafarani (2018). ‘Fake news: a sur- vey of research, detection methods, and opportunities’, arXiv:1812.00315.
Zuckerman, M., B. M. DePaulo and R. Rosenthal (1981). ‘Ver- bal and nonverbal communication of deception’, Advances in Experimental Social Psychology, 14, pp. 1–59.
© 2021 British Academy of Management and Wiley Periodicals LLC.
Social Bots and the Spread of Disinformation in Social Media 1253
NickHajli is Associate Professor in Business at SwanseaUniversity,UK.He has a PhD inManagement fromBirkbeck,University of London.His research has appeared in theBritish Journal of Management, Journal of Business Ethics, Journal of Business Research, Industrial Marketing Management, Annual of Tourism Journal and others. Nick is Associate Editor for the Journal of Business Research and a Senior Editor for Information Technology and People.
Usman Saeed received an MSc in Data Science and Analytics in 2019 from Ryerson University, an MEng in Electrical and Computer Engineering in 2010 from the University of Western Ontario and a BEng in Electronics Engineering in 2007 from NED University. He worked with TELUS for 8 years. Currently, he is working as a data scientist with Rogers Communication, helping them make better business decisions and optimize networks leveraging machine learning and AI. His research interests are in the field of NLP, blockchain, sensor data processing and mobile robotics navigation.
Mina Tajvidi is Assistant Professor/Lecturer inMarketing at QueenMary University of London, UK. Prior to joining Queen Mary, she was a Lecturer in Digital Marketing at Newcastle University, UK. Her research interests are in the areas of service marketing, social media & digital marketing, branding and big data analytics. Her research has been published in leading journals including theBritish Journal of Management, Annals of Tourism Research, Industrial Marketing Management, Journal of Business Ethics, Journal of Business Research, IEEE Transactions on Engineering Management and Computers in Human Behavior.
Farid Shirazi is an Associate Professor at the Ted Rogers School of Information Technology Manage- ment, Ryerson University, Toronto, Canada. He is also a Senior Researcher at the Institute for Inno- vation and Technology Management, a member of the editorial board of Informatics & Telematics, and an Associate Editor of Telematics & Informatics Reports. His research has appeared in the British Journal of Management, Information &Management, Industrial Marketing Management, International Journal of Information Management, Telematics & Informatics and others.
© 2021 British Academy of Management and Wiley Periodicals LLC.
Copyright of British Journal of Management is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.