helpfn

Received February 7, 2020, accepted March 1, 2020, date of publication March 6, 2020, date of current version March 18, 2020.

Digital Object Identifier 10.1109/ACCESS.2020.2978950

Learning Political Polarization on Social Media Using Neural Networks LORIS BELCASTRO 1, RICCARDO CANTINI 1, FABRIZIO MAROZZO 1,2, DOMENICO TALIA 1,2 AND PAOLO TRUNFIO 1,2 1DIMES - University of Calabria, 87036 Rende, Italy 2DtoK Lab Srl, 87036 Rende, Italy

Corresponding author: Fabrizio Marozzo ([email protected])

This work was supported in part by the SMART Project, CUP J28C17000150006, funded by Regione Calabria under Grant POR FESR-FSE 2014-2020, and in part by the ASPIDE Project funded by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 801091.

ABSTRACT Social media analysis is a fast growing research area aimed at extracting useful information from social media platforms. This paper presents a methodology, called IOM-NN (Iterative Opinion Mining using Neural Networks), for discovering the polarization of social media users during election campaigns characterized by the competition of political factions. The methodology uses an automatic incremental procedure based on feed-forward neural networks for analyzing the posts published by social media users. Starting from a limited set of classification rules, created from a small subset of hashtags that are notoriously in favor of specific factions, the methodology iteratively generates new classification rules. Such rules are then used to determine the polarization of people towards a faction. The methodology has been assessed on two case studies that analyze the polarization of a large number of Twitter users during the 2018 Italian general election and 2016 US presidential election. The achieved results are very close to the real ones and more accurate than the average of the opinion polls, revealing the high accuracy and effectiveness of the proposed approach. Moreover, our approach has been compared to the most relevant techniques used in the literature (sentiment analysis with NLP, adaptive sentiment analysis, emoji- and hashtag- based polarization) by achieving the best accuracy in estimating the polarization of social media users.

INDEX TERMS Social media analysis, opinion mining, user polarization, neural networks, sentiment analysis, political events.

I. INTRODUCTION Every day millions of people use social media and produce huge amount of digital data that can be effectively exploited to extract valuable information concerning human dynamics and behaviors. Such data, commonly referred as Big Data, contains valuable information about user activities, interests, and behaviors, which makes it intrinsically suited to a very large set of applications [1]. Big Social Data analysis is a sub- field of Big Data analysis aimed at studying the interactions of users on social media for extracting useful information, such as moods or opinions on topics or events of interest [2].

This paper presents a new methodology, namely IOM- NN (Iterative Opinion Mining using Neural Networks), for estimating the polarization of public opinion on political events characterized by the competition of factions or parties. It can considered as an alternative technique to traditional

The associate editor coordinating the review of this manuscript and

approving it for publication was Vlad Diaconita .

opinion polls, since it is able to capture the opinion of a larger number of people more quickly and at a lower cost. In particular, IOM-NN uses an automatic incremental proce- dure based on feed-forward neural networks for analyzing the posts published by social media users. Starting from a limited set of classification rules, created from a small subset of hashtags that are notoriously in favor of specific factions, our methodology iteratively generates new classification rules. A classification rule allows to determine if a post is in favor of a faction based on the words/hashtags it contains. Then, such rules are used to determine the polarization of social media users - who wrote posts about the political event - towards a faction.

The proposed methodology has been applied to two case studies for analyzing the polarization of a large number of Twitter users during the 2018 Italian general election and the 2016 US presidential election. The results obtained by IOM-NN have been compared to opinion polls collected before voting and the most relevant techniques used in

VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 47177

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

the literature (i.e., sentiment analysis with NLP [3], adap- tive sentiment analysis [4], emoji-based polarization [5], hashtag-based polarization [6]). The results achieved by IOM-NN are very close to the real ones and more accu- rate than opinion polls and other relevant techniques, reveal- ing the high accuracy and effectiveness of the proposed approach. For example, considering the 2018 Italian gen- eral election and the four parties that received the highest number of votes (M5S, PD, LEGA, FI), IOM-NN achieved a mean average error (MAE) of 1.13 percentage points and a log accuracy ratio (LogAcc) very close to 1. Opinion polls achieved a MAE of 3.74 percentage points and a LogAcc of 0.81. Compared with the other existing tech- niques, IOM-NN turned out to be the most accurate in fore- casting the winning candidate. For example, considering the 2016 US presidential election, IOM-NN has been able to correctly identify the winning candidate in 8 out of 10 states, while the other techniques identified the winner in up to 6 out of 10 states.

Compared to existing techniques, our methodology includes the following innovative aspects: i) it allows to classify a much high number of users, which in our case studies results to be ten times larger than that involved in traditional opinion polls; ii) it measures the political consen- sus of a faction considering the number of users supporting it, without being influenced by users who published a large volume of posts; iii) it is able to process posts written in any language without the need to use dictionaries or translation systems; iv) at the best of our knowledge, it is the first methodology in this research field that exploits an iterative learning approach to increase the amount of classified data; v) collected data has been statistically validated for assessing the representativeness of users involved in the analysis.

This manuscript significantly extends a previous work [7] in the following main aspects: i) it provides an in-depth definition of the steps of the methodology and a formal description through pseudo-code of the algorithms used for classifying the posts and predicting the users’ polarization (see Sections III-B and III-C); ii) it presents and discusses a new case study on which the methodology has been applied and tested (see Section IV); iii) it includes the evaluation of the statistical significance of the collected data for the two case studies; and iv) it reports more extensive and detailed tests and comparisons with relevant techniques used in the literature (see Section IV).

The remainder of the paper is organized as follows. Section II discusses related work. Section III describes the proposed methodology. Section IV presents the case studies and Section V concludes the paper.

II. RELATED WORK In recent years, social media analysis is arousing great interest in various scientific fields, such as sociology, political sci- ence, linguistics, and computer science [8]. In this section, we focus on the main techniques and algorithms proposed for measuring public opinion and predicting election results

through social media. As suggested in [9] and [10], the exist- ing techniques can be divided into three main categories: volume-based, sentiment-based and network-based. For each category, the main proposed solutions and their differences with respect to our technique are discussed. Volume-based techniques counts the number of mentions

(e.g., posts, likes, retweets) related to a candidate/party for predicting the election results. In many cases, such tech- niques analyze social media data for predicting the outcome of an election. For example, Gaurav et al. [11] proposed a technique based on moving average aggregate probability, which infers the results of an election by counting how many times a candidate’s name is mentioned in tweets. Tumasjan et al. [12] used micro-blogging data for under- standing how people express their political orientation, showing a high closeness between the volume of tweets mentioning a party and the election results. Burnap et al. [13] analyzed the volume of mentions for calculating an overall score for each party.

Unlike volume-based techniques, which consider the num- ber of posts in favor of a faction, our technique takes into account the number of users supporting a faction. In this way, IOM-NN obtains more accurate results since it is not influenced by users who published a large number of posts. Sentiment- or opinion-based techniques exploit natural

language processing (NLP) or text mining algorithms for understanding the opinion of users towards political candi- dates or parties. Such techniques result to be more advanced than the volume-based ones as they analyze the textual con- tent of posts to calculate a score.

The techniques based on natural language processing consider the hierarchical structure of a text to understand its meaning and sentiment. For example, Oikonomou and Tjortjis [3] exploited Textblob,1 a Python library for nat- ural language processing, to predict the outcome of USA presidential elections in three states of interest (i.e., Florida, Ohio and North Carolina). Wong et al. [14] combined convex optimization techniques with SentiStrength,2 a lexicon-based sentiment analysis tool, for modeling the political behaviors of users by analyzing tweets and retweets. Alashri et al. [15] analyzed Facebook posts about the 2016 US presidential election with CoreNLP3 [16], one of the most popular tool for natural language processing, to calculate a score for each political candidate.

Techniques based on text mining discover the sentiment of a text by considering only the words it contains without analyzing its structure. For example, El Alaoui et al. [4] pro- posed an adaptive sentiment analysis approach that generates dictionaries from tweets classified as positive/negative for the different factions. Such dictionaries are then used to calculate a score for each faction. Similarly, Marozzo and Bessi [6] calculated a polarization score for each faction by considering

1https://textblob.readthedocs.io/ 2http://sentistrength.wlv.ac.uk/ 3https://stanfordnlp.github.io/CoreNLP/

47178 VOLUME 8, 2020

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

FIGURE 1. Execution flow of IOM-NN.

only the hashtags in tweets labeled as positive. Chin et al. [5] exploited the emojis contained in a post to determine its sentiment (e.g., positive or negative). Other studies exploited machine learning techniques for discovering the political ori- entation of users, such as classification models based on the Naïve Bayes algorithm [17] or logistic regression [18].

IOM-NN is a text mining technique that uses bag-of-words and neural networks to classify posts, and consequently users who wrote such posts about a political event. Compared to existing text mining techniques, its iterative approach allows to greatly increase the number of classified posts, while the use of neural networks permit to automatically discover clas- sification rules with a high level of accuracy. With regard to NLP techniques, it is worth noting that their usability and accuracy depend on the specific tool used and the supported languages. In fact, the most popular tools for natural language processing (e.g., CoreNLP) support sentiment analysis only for English texts. Instead IOM-NN classifies users by ana- lyzing the words/hashtags contained in the posts that can be written in any language, without using dictionaries or trans- lation systems. Network-based techniques analyze the network structure

of social media users, which support or discuss about cer- tain candidates or parties, for understanding the dynamics of public opinion. Such analysis can provide useful insights for estimating the standing of political events or identifying the opinion leaders on a social media platform [19]. In fact, some studies have demonstrated a relation between the centrality of political candidates on social networks and their electoral consensus [20], [21]. However, it should be noticed that such techniques require the use of specific data that represents the social network structure, which is often visualized through graphs or sociograms. In our study, we collected and analyzed tweets containing specific keywords or hashtags on the polit- ical event under analysis, without capturing the structure of the related social network. For this reason, in this paper we cannot make a comparison with network-based techniques.

Some studies highlighted the issues related to the use of social media data for predicting the outcome of political events, which are language barrier, misclassification, data imbalance and reliability [10]. During the design of IOM-NN we faced such issues by proposing the following solutions: • Language barrier. Our technique classifies users by analyzing the words/hashtags contained in their posts, regardless of the language used to write them.

• Misclassification. Starting from a limited set of classifi- cation rules, created from a small set of hashtags that are

notoriously in favor of specific factions, the methodol- ogy iteratively generates new classification rules.

• Data imbalance. To avoid the learning process being biased towards majority classes, a random under- sampling approach is used to balance the dataset at each training phase (see Algorithm 1).

• Data reliability. The statistical significance of the col- lected data has been evaluated for assessing the rep- resentativeness of users, i.e., understanding whether they can be considered voters in the political event under analysis.

III. PROPOSED METHODOLOGY (IOM-NN) As mentioned in Section I, IOM-NN is a methodology for estimating the polarization of public opinion during a polit- ical event, which is characterized by the rivalry of different factions. As shown in Figure 1, the proposed methodology consists of three main steps:

1) Collection of posts: posts are collected by using a set of keywords related to the selected political event (see Section III-A).

2) Classificationofposts: the collected posts are then clas- sified by using an incremental procedure implemented through neural networks (see Section III-B).

3) Polarization of users: the classified posts are analyzed for determining the polarization of users towards a faction (see Section III-C).

TABLE 1. Meaning of the most important symbols used in the proposed methodology.

For each step, a formal description and practical examples are provided in the following sections. For the sake of clarity, Table 1 reports the meaning of the main symbols used to describe the different steps.

VOLUME 8, 2020 47179

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

A. COLLECTION OF POSTS A political event E is characterized by the rivalry of different factions F ={f1, f2, . . . , fn}. Examples of political events and relative factions are: i) municipal election, in which a faction supports a mayor candidate; ii) parliament election, in which a faction supports a party; iii) presidential election, in which a faction supports a presidential candidate. The posts are collected by using the keywords that people commonly use to refer the political event E on social media. Such keywords K can be divided in two groups:

- Kcontext, which contains generic keywords that can be associated to E without referring to any specific faction in F.

- K⊕F = K ⊕

f 1∪. . .∪K ⊕

fn , where K ⊕

fi contains the keywords used for supporting fi ∈ F (positive faction keywords).

The keywords in K are given as input to public APIs provided by social media platforms, which permit to col- lect posts containing one or more keywords. Posts are not collected in real time, but downloaded at a given time after their publication (e.g., 24 hours). In this way, we are able to get some statistics related to the popularity of a post (e.g., number of shares, number of likes). Since data collec- tion is usually a continuous process, new keywords can be dis- covered and integrated in K during the collection procedure. It is important to highlight that obtaining a representative collection of posts depends on two factors: i) the quality and the number of keywords used; ii) the amount of data that can be downloaded from social media. Regarding the latter factor, it is increasingly difficult to obtain complete data from social media platforms due to the restrictions introduced for protecting the privacy of users.

The collected posts are pre-processed before the analysis. In particular, they are modified and filtered as follows:

• The text of posts is normalized by transforming it to lowercase and replacing accented characters with reg- ular ones (e.g., IOVOTOSI or iovotosí → iovotosi).

• Words are stemmed for allowing matches with declined forms (e.g., vote or votes or voted → vot).

• Stop words are removed from text by using preset lists. • All the posts written in a language different from the one(s) spoken in the nation(s) hosting the considered political event are filtered out.

The output of this step is a collection of posts P. Figure 2 shows an example of how posts are collected using keywords about the 2016 US presidential election. Some of these key- words are generic (e.g., election2016), and others are used to support a specific candidate (e.g., #imwithher for Clinton and #votetrump for Trump).

Before the analysis, the statistical significance of the col- lected data has to be evaluated. We studied the age, gender and geographical distribution of social media users who gen- erated such data. The aim is to assess the users’ represen- tativeness by understanding whether they can be considered voters of the political event under analysis (more details in Sections IV-A.1 and IV-B).

FIGURE 2. Example of how the collection of posts step works.

Algorithm 1 Classification of the Posts

Input : Set of posts P, set of positive faction keyword K⊕F , threshold th, minimum increment eps, maximum number of iterations maxiters

Output: Classified posts C 1 C0 ←∅; 2 M0 ← textualModel.build(K⊕F ); 3 for p ∈ P do 4 vb ← classify(M

0, p); 5 if sum(vb) = 1 then 6 f ← argmax(v); 7 C0 ← C0 ∪〈p, f 〉;

8 C ← C0; 9 N0 ← P\C0; 10 for i = 1; i <= maxiters; i++ do 11 Ci ←∅; 12 Mi ← neuralNetwork.train(C0 ∪ . . . ∪Ci−1); 13 for p ∈ N i−1 do 14 vp ← classify(Mi, p); 15 if max(vp) > th then 16 f ← argmax(v); 17 Ci ← Ci ∪〈p, f 〉;

18 C ← C ∪Ci; 19 N i ← N i−1 \Ci;

20 if |C i |

|N i−1| < eps∨ |C

i |

|N i−1| > 1− eps then

21 break

22 return C

B. CLASSIFICATION OF POSTS Algorithm 1 shows the pseudo-code used for classifying the posts. The input is composed of: the posts P generated in the previous step, the set of positive faction keywords K⊕F , the maximum number of iterations maxiters, the minimum increment of the classified posts eps at each iteration, and a threshold th. The output is a collection of posts C that have been classified in favor of a faction.

The algorithm is divided in two parts. The fist part (lines 1-9) performs the preliminary iteration (iteration 0). At this iteration, IOM-NN exploits the set of positive faction keywords (K⊕F ) for classifying a part of the posts. Specifi- cally, it classifies a post in favor of a faction if it contains only positive keywords for such faction. In general, at the end of this iteration, a small amount of posts are classified, since not all users use keywords in K⊕F for declaring their support to factions. The second part (lines 10-21) iteratively generates new classification rules for classifying other posts.

47180 VOLUME 8, 2020

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

At each iteration, such rules are inferred by exploiting the posts that have been classified at the previous iterations. In the following of this section, we discuss in detail the code of Algorithm 1.

The algorithm initializes an empty set C0 for storing the classified posts and builds a classification model M0 based on the positive faction keywords K⊕F (lines 1-2). The model defines a set of rules for calculating a binary vector vb, where vb[i] is 1 if p contains at least one keyword from K

⊕

fi , 0 otherwise. The algorithm iterates (lines 3-7) on each post p in P performing the following operations:

- classifies p using M0, which produces a vector vb (line 4);

- if p is in favor of a single faction f (lines 5-6), the clas- sified post (i.e., a pair 〈p, f 〉) is added to C0 (line 7).

At the end of iteration 0, the set of classified posts C0

is stored in C (line 8). The set of unclassified posts (N0) is obtained as the difference between P and C0 (lines 9). It is important to note that the number of keywords has to be balanced among the different factions for avoiding a classification process biased towards a faction.

The second part of the algorithm (lines 10-21) performs at most maxiters iterations. Specifically, at the i-th iteration, the following operations are performed:

- It initializes an empty set Ci for storing the classified posts at i-th iteration (line 11).

- It builds a classification model Mi by training a neural network using the classified posts at previous iterations C0 ∪ . . . ∪Ci−1 (lines 12). The training set is balanced by using a random under-sampling approach to avoid a learning process biased towards majority classes.

- For each unclassified post at the previous iteration N i−1

(line 13), the algorithm classifies p using Mi, which produces a vector of probabilities vp (line 14), where vp[i] is the probability that p supports fi. If the maximum value of vp is greater than the given threshold th, the post is assigned to the most likely faction f (lines 15-16) and added to Ci (line 17).

- The set of classified posts Ci is added to C (line 18), and the unclassified posts N i are obtained as difference between N i−1 and Ci (line 19).

- If the ratio between the size of Ci and the size of N i−1

is lower than eps or greater that 1 − eps, then it breaks the loop (lines 20-21).

Finally, the algorithm returns the dictionary C containing all the posts classified at the various iterations (line 22). Since the parameters of the neural network are randomly initialized, IOM-NN repeats the post classification phase with a new random seed for nseeds times, in order to reduce the risk of getting stuck in local minima or saddle points.

Figure 3 shows how the post classification algorithm (Algorithm 1) works starting from a set of posts P. At the iteration 0, the classification model M0 is created using the faction keywords KF. This model is used to classify P, which generates two subsets for classified (C0) and unclassified

FIGURE 3. Example of the classification of posts algorithm terminating in three iterations.

posts (N0) respectively. At iteration 1, a new model M1 is trained using C0 and is used to classify the unclassified posts generated at the previous iteration (N0). The classification process splits N0 in two new subsets: C1 for classified posts and N1 for unclassified ones. Then, at the i − th iteration, the model Mi is trained using C0 ∪ . . .∪Ci−1. For example, at iteration 2 the model M2 is trained using C0 ∪ C1. The process is repeated in subsequent iterations until the ratio between the size of Ci and the size of N i−1 is lower than eps or greater that 1 − eps. At the end, the whole set of classified posts is obtained as the union of the Ci produced at each iteration, while the remaining posts (N2) are classified as neutrals.

Table 2 shows an example of post classification on ten tweets about the 2016 US presidential election. The input of the algorithm is composed of a set of tweets regarding the political event and a set of faction keywords K⊕F : • K⊕Clinton={#voteHillary, #imwithher, #strongertogether, #hillary2016}

• K⊕Trump = {#voteTrump, #maga, #americafirst, #wakeu- pamerica}

At iteration 0, K⊕F is used to generate M 0, which allows to

classify 5 tweets. At iteration 1, classified tweets at iteration 0 are used to train M1. This model generates new classifica- tion rules, such as: • since Donald Trump has been accused of sexual assault by some women, tweets with keywords #sex and #woman are classified in favor of Clinton;

• similarly, since Hillary Clinton contravenes the federal laws by using personal email account for government business, tweets with keywords email and #hillary are classified in favor of Trump.

At iteration 2, the algorithm learns other classification rules about immigration, a topic on which the two candidates had an opposite opinion. The iterative learning process ends when the algorithm is no longer able to generate new classification rules and therefore to classify new tweets.

VOLUME 8, 2020 47181

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

TABLE 2. Example of how the classification of posts algorithm works.

Algorithm 2 Prediction of User Polarization Input : Classified posts C, filtering function filter, filtering

function parameters parf , polarization function polarize, polarization function parameters parp.

Output: Classified users U, faction score S 1 CU ← aggregateByUser(C); 2 U ←∅; 3 S ←∅; 4 for 〈u, Pu〉 ∈ CU do 5 if filter(〈u, Pu〉, parf ) then 6 vus ← polarize(Pu, parp); 7 U ← U ∪〈u, vus〉;

8 for 〈u, vus〉 ∈ U do 9 S ← S + vus ;

10 S ← S/sum(S); 11 return U, S

C. POLARIZATION OF USERS Algorithm 2 shows the pseudo-code of the algorithm used for determining the polarization of users. The input is a collection of classified posts C (i.e., output of Algorithm 1), a filtering function filter and its parameters parf , and a polarization function polarize and its parameters parp. The output is com- posed of a collection of classified users U and a faction score (S) containing the polarization percentages of each faction.

As first step, the classified posts are aggregated by user to produce a dictionary (CU), which contains the list of classi- fied posts Pu for each user u (line 1). Two empty variables are initialized for storing the output (lines 2-3).

On each pair 〈u, Pu〉 of CU , the algorithm performs the following operations (lines 4-7):

- It filters out all the pairs that do not match the criteria defined by the filter function (line 5). For example,

users who published a number of posts below a given threshold are skipped.

- Using the classified posts Pu, it computes vus a vector containing the score of user u for each faction (line 6). The score vector is calculated by using the function polarize.

- It adds the pair 〈u, vs〉 to U (line 7). Then, the algorithm calculates the overall faction score

S as the normalized sum of the user vector scores 〈u, vus〉 (lines 8-10). Finally, the two output are returned (line 11).

The filter and polarize functions, used for analyzing the data collected for our case studies (see Section IV), have been configured as follows. Specifically, a user u is considered only if he/she fulfills the following criteria: i) u posted at least minPosts on the political event of interest; ii) it exists a faction f for which u has published more than 2/3 of his/her posts. For each user u, the polarize function returns a vector score as follows: the percentage of posts written by u in favor of preferred faction f , 0 for the other factions.

FIGURE 4. Example of how the user polarization algorithm works.

Figure 4 shows how the user polarization algorithm (Algorithm 2) works on the classified posts shown in Table 2. For each user, the posts if favor of Clinton and Trump

47182 VOLUME 8, 2020

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

are counted. Users who fulfill the criteria of filter function are considered and added to the set of classified users U. Then U is combined and normalized to obtain the vector S containing the overall polarization percentages.

IV. CASE STUDIES In this section we describe and analyze two case studies: the 2018 Italian general election and 2016 US presiden- tial election. In both case studies, for each faction fi we defined three set of keywords K⊕fi , K

� fi and K

} fi that are

respectively positive, negative and neutral keywords for fac- tion fi. For example, for the Movimento 5 Stelle (M5S) faction in the 2018 Italian general election, K⊕M5S contains keywords used to clearly support M5S party (e.g., #iovo- toM5S), K�M5S contains keywords to speak negatively about M5S (e.g., #maiM5S), K}M5S contains neutral keywords for M5S (e.g., m5s or movimento5stelle). As described in Section III, IOM-NN exploits only positive

faction keywords (K⊕fi ) for classifying posts and then for determining the polarization of users. For evaluating the accu- racy of IOM-NN, we carried out an extensive comparison with the most relevant techniques used in literature:

1) Sentiment analysis with NLP [3], [15]. For each post, we used CoreNLP [16] for calculating a sentiment score that ranges from 0 (very negative) to 4 (very positive). The neutral keywords (K}fi ) are then used for grouping posts and calculating an overall score for each faction.

2) Adaptive sentiment analysis [4]. Starting from the positive and negative keywords of each faction (K⊕fi and K

� fi ), this technique generates two word-

polarity dictionaries, which are built from a set of posts containing such positive and negative keywords. Also in this case, a score for each faction is returned.

3) Emoji-based polarization [5]. This technique groups the posts of each faction by using keywords (K}fi ), then classifies their sentiment by using emojis and returns a score for each faction.

4) Hashtag-based polarization [6]. The posts are classi- fied as in favor of a given faction based on the positive faction keywords (K⊕fi ). Then the posts are aggregated by users and the polarization of each user is computed.

To allow a direct comparison with the real percentages, the results obtained by the different techniques have been normalized with respect to the sum of the real ones.

A. 2018 ITALIAN GENERAL ELECTION Here we discuss the case study carried out to analyze the polarization of a large number of Twitter users during the 2018 Italian general election. Twitter users have been classi- fied using the polarization rules extracted from our methodol- ogy and the results have been compared to: i) official results; ii) main opinion polls collected before voting; and iii) other techniques present in the literature.

Italians voted to elect 630 deputies and 315 senators of the XVIII legislature: the results decreed the center-right

coalition as the most voted, with about 37% of votes, while the most voted list was the Movimento 5 Stelle, which received over 32% of votes. The electorate was composed of 50,782,650 voters for the Chamber of Deputies and 46,663,202 for the Senate,4 with a turnout of about 73%, the lowest in Italian republican history.

In order to assess the validity of the proposed methodology, the analysis we carried out focused on the four most success- ful political factions, in decreasing order of consensus: M5S (Movimento 5 Stelle), PD (Partito Democratico), LEGA, FI (Forza Italia). In the following, we show how the classifica- tion model has been trained and discuss the main achieved results.

1) MODELS TRAINING AND ITERATION-LEVEL RESULTS IOM-NN has been used to classify 60,782 tweets posted by 21,883 users from February 1, 2018 to March 3, 2018 (the day before the election). By following the approach described in [6] we can assess that collected data is statistically signifi- cant for the event under analysis, since: • All the tweets under analysis have been written in Italian, that means they have the lang field set to it (Italian). With very few exceptions, the Italian language is used only by Italians people who reside in Italy or abroad.5

• 92% of the users who set a location in their profile, specified a region in Italy. Moreover, there is a strong correlation between the number of users that can be assigned to a region and the number of people actually living in that region according to official statistics (the Pearson correlation coefficient is equal to 0.8 with a confidence interval of 99%).

• About 98% of the Italian social media users are adults and equally divided by gender (51.2 females and 48.8 males)6.

IOM-NN exploits the following positive faction keywords for analyzing the collected data: • K⊕M5S = {#iovotom5s,#m5salgoverno, #dimaiopresi- dente}

• K⊕PD = {#sceglipd,#iovotopd,#pdvinci}

• K⊕LEGA = {#4marzovotolega,#iovotolega, #salvini- premier}

• K⊕FI = {#berlusconipresidente,#votoforzaitalia, #4mar- zovotoforzaitalia}

The threshold th and the minimum increment eps have been set to 0.9 and 5% respectively. In our test, the post classification algorithm terminated in 4 iterations by anno- tating 23,997 tweets, which represents about 39.5% of the total. Table 3 shows the obtained results at each iteration by specifying the number of classified and unclassified tweets, the ratio |C

i |

|N i−1| and the accuracy of the neural network.

4http://www.interno.gov.it/it/notizie/elezioni-2018-come-vota-corpo- elettorale-tessera-elettorale (in Italian)

5https://en.wikipedia.org/wiki/Italian_language 6https://wearesocial.com/it/digital-2019-italia

VOLUME 8, 2020 47183

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

TABLE 3. Partial results for each iteration achieved by IOM-NN (2018 Italian general election).

TABLE 4. Obtained percentages and accuracy evaluation on the 2018 Italian general election dataset.

2) POLARIZATION OF USERS AND FINAL RESULTS The algorithm described in Section III-C has been used for analyzing the users who have written the 23,997 classified tweets so as to determine their polarization degree towards the considered factions. The first three rows of Table 4 shows a comparison between the official results, the average of the latest polls and the percentages obtained by IOM- NN. We evaluated the accuracy through different statisti- cal indexes, comparing the obtained results with the latest opinion polls published before the elections. Considering the four most supported parties, our methodology obtained the following approval percentages: M5S 31.64%, PD 19.89%, LEGA 18.45%, and FI 12.80%. These results are extremely close to the real ones (i.e., M5S 32.68%, PD 18.72%, LEGA 17.37%, FI 14.01%), even more than the average of polls. In addition, the obtained results are characterized by very good values of log accuracy ratio, as well as a negligible value of mean percentage and absolute errors. In particu- lar, our methodology achieved a mean average error (MAE) of 1.13 percentage points and a log accuracy ratio (LogAcc) very close to 1. On the other hand, opinion polls achieved a MAE of 3.74 percentage points and a LogAcc of 0.81, which confirm the ability of the proposed methodology to fore- cast election results. Figure 5 shows an info-graphic about the comparison of the real percentages, opinions polls and obtained results.

Table 4 also presents the results obtained by the other techniques in the literature (rows 4-7). These techniques have been configured with the positive faction hashtags used by IOM-NN (see K⊕M5S, K

⊕

PD, K ⊕

LEGA and K ⊕

FI) and the following negative and neutral faction keywords: • K�M5S ={#nom5stelle, #rimborsopolim5s, #maim5s} and K}M5S = {m5s, movimento5stelle, dimaio}

• K�PD ={#nonvotopd, #maipd, #bastapd} and

K}PD = {pd, partitodemocratico, renzi}

FIGURE 5. Comparison among real percentages, opinions polls and IOM-NN results (2018 Italian general election).

• K�LEGA ={#maiconsalvini, #iononvotolega} and

K}LEGA ={lega, salvini, leganord}

• K�FI ={#maipiuberlusconi, #stopberlusconi} and

K}FI = {forzaitalia, berlusconi} Compared to such techniques, IOM-NN turned out to be

the most accurate in estimating the voting percentages, out- performing the competitors in terms of achieved LogAcc, MAPE and MAE. Compared to the emoji- and hashtag- based techniques, IOM-NN is able to classify a much greater number of tweets and users, which ensures greater statistical representativeness of data and robustness of results.

It should be noticed that, differently from other techniques (i.e., sentiment analysis with NLP and adaptive sentiment analysis), IOM-NN is not volume-based. This means that, since it gives the same weight to each user regardless of the number of published posts, the results obtained are not influenced by users who published a large number of posts.

47184 VOLUME 8, 2020

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

TABLE 5. Obtained percentages and accuracy evaluation on the 2016 US presidential election dataset. For each technique, when the winning candidate is correctly identified, the percentage is written in bold.

Since CoreNLP provides a well-trained model for sentiment analysis only for the English language, all the tweets down- loaded in Italian have been translated in English before being processed.

B. 2016 US PRESIDENTIAL ELECTION After the presentation of the Italian use case, here we discuss the analysis we carried out on the 2016 US presidential elec- tion, which was characterized by the rivalry between Hillary Clinton and Donald Trump.

The analysis has been performed on data collected for ten US Swing States: Colorado, Florida, Iowa, Michigan, Ohio, New Hampshire, North Carolina, Pennsylvania, Virginia, and Wisconsin. Swing states are those characterized by greater political uncertainty, in which neither major political party holds a lock on the outcome of presidential elections. These states are considered of strategic importance, as their votes have a high probability of being the deciding factor in a presidential election. For each state, data have been collected through the standard Search Twitter API, which allows for collecting tweets published in a given area or place. Overall about 2.5 million of tweets, posted by 521,291 users, have been collected from October 10, 2016 to November 7, 2016 (the day before the election). From such data we filtered out all the tweets posted by users with a not defined loca- tion or with a location that does not belong to any of the considered states. Filtered data (818,403 tweets posted by 141,959 users) are statistically significant for the event under analysis, since: • All the tweets under analysis have the lang field set to en (English).

• For each state, there is a strong correlation between the number of analyzed users and the number of people actually living in that state according to official statistics (the Pearson correlation coefficient is equal to 0.95 with a confidence interval of 99%).

• About 94% of the social media users in USA are adults (at least 18 years old) and almost equally divided by gender (42.7% females and 57.3% males).7

The following keywords have been used in our experi- ments: • K⊕Clinton={#voteHillary, #imwithher, #strongertogether, #hillary2016},

• K�Clinton = {#neverhillary, #lockherup}

• K}Clinton = {clinton, hillary, democrats, dems}

• K⊕Trump = {#voteTrump, #maga, #americafirst, #wakeu- pamerica}

• K�Trump = {#nevertrump, #dumpfortrump}

• K}Trump = {trump, donald, republicans, gop} Table 5 shows the results obtained using IOM-NN in com-

parison with the real voting percentages, the main opinion polls, and the other related techniques. For each state in the table, we reported the results obtained by the two candi- dates, where ‘‘C’’ stands for Clinton and ‘‘T’’ for Trump. In addition, for the winning candidate that has been correctly identified, the value is written in bold. As shown in Figure 6, IOM-NN is able to correctly identify the winning candidate in 8 out of 10 cases, outperforming the opinion polls that correctly classifies 6 out 10 states.

Also in comparison with the other techniques, IOM-NN turned out to be the most accurate in discovering the win- ning candidate. In fact, emoji- and hashtag-based techniques classified a much smaller amount of tweets and correctly identified the winning candidate in 6 and 3 cases respectively. The results of the adaptive sentiment analysis were very poor, since it correctly identified the winning candidate in only one case, while sentiment analysis with NLP produced right predictions in 4 out 10 cases. Compared to IOM-NN,

7https://www.statista.com/statistics/376128/facebook-global-user-age- distribution/

VOLUME 8, 2020 47185

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

FIGURE 6. Comparison among the real winning candidate and that identified by IOM-NN and opinions polls. The Democratic Donkey symbolizes the party of Hillary Clinton, while the Republican Elephant that of Donald Trump.

some techniques (i.e., adaptive sentiment analysis and opinion polls) achieved slightly better results in terms of log accuracy ratio, MAPE, and MAE, because their predictions are quite balanced by assigning almost the same score to the two candidates in the different states.

V. CONCLUSION This paper proposes a methodology, named IOM-NN, for estimating the polarization of public opinion regarding political events characterized by the competition of fac- tions or parties. The designed methodology uses an auto- matic incremental procedure based on feed-forward neural networks for analyzing the posts published by social media users. IOM-NN can be considered an alternative technique to traditional opinion polls since it is able to capture the opinion of a larger number of people more quickly and at a lower cost. In addition, it can capture the public opinion for topics perceived as embarrassing or offensive, for which people are reluctant to declare their true opinion during the polls.

IOM-NN has been validated through two case studies that analyzed the polarization of a large number of Twitter users during the 2018 Italian general election and 2016 US pres- idential election. The achieved results are very close to the real ones and more accurate than the average of the opinion polls, thus revealing the high accuracy and effectiveness of the proposed approach. Moreover, our approach has been compared to the most relevant techniques used in the liter- ature (sentiment analysis with NLP, adaptive sentiment anal- ysis, emoji- and hashtag- based polarization). Results show that IOM-NN achieved the best accuracy in estimating the polarization of social media users.

As future work, IOM-NN can be adapted to pro- cess real-time data, also coming from different sources (e.g, e-commerce and news sites blogs). Furthermore, its effectiveness can be evaluated also in other application domains, such as reputation evaluation of companies and competitive product analysis.

REFERENCES [1] L. Belcastro, F. Marozzo, and D. Talia, ‘‘Programming models and systems

for big data analysis,’’ Int.J.Parallel,EmergentDistrib.Syst., vol. 34, no. 6, pp. 632–652, Nov. 2019.

[2] D. Talia, P. Trunfio, and F. Marozzo, Data Analysis in the Cloud. Amster- dam, The Netherlands: Elsevier, 2015.

[3] L. Oikonomou and C. Tjortjis, ‘‘A method for predicting the winner of the USA presidential elections using data extracted from Twitter,’’ in Proc. South-Eastern Eur. Des. Autom., Comput. Eng., Comput. Netw. Soc. Media Conf. (SEEDA-CECNSM), Sep. 2018, pp. 1–8.

[4] I. El Alaoui, Y. Gahi, R. Messoussi, Y. Chaabi, A. Todoskoff, and A. Kobi, ‘‘A novel adaptable approach for sentiment analysis on big social data,’’ J. Big Data, vol. 5, no. 1, p. 12, Dec. 2018.

[5] D. Chin, A. Zappone, and J. Zhao, ‘‘Analyzing Twitter sentiment of the 2016 presidential candidates,’’ Amer. J. Sci. Res., vol. 1, pp. 128–137, Jan. 2016.

[6] F. Marozzo and A. Bessi, ‘‘Analyzing polarization of social media users and news sites during political campaigns,’’ Social Netw. Anal. Mining, vol. 8, no. 1, p. 1, Dec. 2018.

[7] L. Belcastro, R. Cantini, F. Marozzo, D. Talia, and P. Trunfio, ‘‘Discovering political polarization on social media: A case study,’’ in Proc. 15th Int. Conf. Semantics, Knowl. Grids, Guangzhou, China, 2019, pp. 1–8.

[8] M. Bilal, A. Gani, M. I. U. Lali, M. Marjani, and N. Malik, ‘‘Social profiling: A review, taxonomy, and challenges,’’ Cyberpsychology, Behav., Social Netw., vol. 22, no. 7, pp. 433–450, Jul. 2019.

[9] K. Jaidka, S. Ahmed, M. Skoric, and M. Hilbert, ‘‘Predicting elections from social media: A three-country, three-method comparative study,’’ Asian J. Commun., vol. 29, no. 3, pp. 252–273, May 2019.

[10] M. Bilal, A. Gani, M. Marjani, and N. Malik, ‘‘Predicting elections: Social media data and techniques,’’ in Proc. Int. Conf. Eng. Emerg. Tech- nol. (ICEET), Feb. 2019, pp. 1–6.

[11] M. Gaurav, A. Srivastava, A. Kumar, and S. Miller, ‘‘Leveraging candidate popularity on Twitter to predict election outcome,’’ in Proc. 7th Workshop Social Netw. Mining Anal. (SNAKDD). New York, NY, USA: ACM, 2013, p. 7.

[12] A. Tumasjan, T. O. Sprenger, P. G. Sandner, and I. M. Welpe, ‘‘Pre- dicting elections with twitter: What 140 characters reveal about political sentiment,’’ in Proc. 4th Int. AAAI Conf. Weblogs Social Media, 2010, pp. 178–185.

[13] P. Burnap, R. Gibson, L. Sloan, R. Southern, and M. Williams, ‘‘140 characters to victory?: Using Twitter to predict the UK 2015 general election,’’ Electoral Stud., vol. 41, pp. 230–233, Mar. 2016.

[14] F. M. F. Wong, C. W. Tan, S. Sen, and M. Chiang, ‘‘Quantifying political leaning from tweets, retweets, and retweeters,’’ IEEE Trans. Knowl. Data Eng., vol. 28, no. 8, pp. 2158–2172, Aug. 2016.

[15] S. Alashri, S. S. Kandala, V. Bajaj, R. Ravi, K. L. Smith, and K. C. Desouza, ‘‘An analysis of sentiments on Facebook during the 2016 U.S. Presidential election,’’ in Proc. IEEE/ACM Int. Conf. Adv. Social Netw. Anal. Mining (ASONAM), Aug. 2016, pp. 795–802.

[16] C. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. Bethard, and D. McClosky, ‘‘The stanford CoreNLP natural language processing toolkit,’’ in Proc. 52nd Annu. Meeting Assoc. Comput. Linguistics, Syst. Demonstrations, 2014, pp. 55–60.

[17] M. K. Olorunnimbe and H. L. Viktor, ‘‘Tweets as a vote: Exploring political sentiments on twitter for opinion mining,’’ in Foundations of Intelligent Systems, F. Esposito, O. Pivert, M.-S. Hacid, Z. W. Rás, and S. Ferilli, Eds. Cham, Switzerland: Springer, 2015, pp. 180–185. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-319-25252-0_19

47186 VOLUME 8, 2020

L. Belcastro et al.: Learning Political Polarization on Social Media Using Neural Networks

[18] A. Bovet, F. Morone, and H. A. Makse, ‘‘Predicting election trends with Twitter: Hillary clinton versus Donald Trump,’’ CoRR, vol. abs/1610.01587, pp. 1–12, Oct. 2016.

[19] F. Bodendorf and C. Kaiser, ‘‘Detecting opinion leaders and trends in online social networks,’’ in Proc. 2nd ACM Workshop Social Web Search Mining (SWSM). New York, NY, USA: ACM, 2009, pp. 65–68.

[20] E. M. Hafner-Burton and A. H. Montgomery, ‘‘Centrality in politics: How networks confer power,’’ Southern Illinois Univ., Carbondale, IL, USA, Tech. Rep. 9, 2010.

[21] P. R. Miller, P. S. Bobkowski, D. Maliniak, and R. B. Rapoport, ‘‘Talking politics on Facebook: Network centrality and political discussion prac- tices in social media,’’ Political Res. Quart., vol. 68, no. 2, pp. 377–391, Jun. 2015.

LORIS BELCASTRO received the Ph.D. degree in information and communication engineering from the University of Calabria. In 2012, he held a scholarship at the Institute of High- Performance Computing and Networking, Italian National Research Council (ICAR–CNR). He is currently a Research Fellow of computer engi- neering with the University of Calabria, Italy. His research interests include cloud computing, social media and Big Data analysis, distributed knowledge discovery, and data mining.

RICCARDO CANTINI graduated in computer sci- ence, in 2019. He is currently pursuing the Ph.D. degree in information and communication tech- nologies with the University of Calabria. His research interests include social media and Big Data analysis, deep learning, edge/fog computing, distributed knowledge discovery, and data mining.

FABRIZIO MAROZZO received the Ph.D. degree in systems and computer engineering from the University of Calabria. From 2011 to 2012 he visited the Barcelona Supercomputing Center for a research internship with the Grid Computer Research Group with the Computer Sciences Department. He is currently an Assistant Profes- sor of computer engineering with the University of Calabria. He is serving as an Associate Editor for IEEE ACCESS and the International Journal of

Intelligent Systems Technologies and Applications. His research interests include distributed systems, data mining, cloud computing, social media and Big Data analysis, and peer-to-peer networks.

DOMENICO TALIA is currently a Professor of computer engineering with the University of Cal- abria and an Adjunct Professor with Fuzhou Uni- versity. His research interests include parallel and distributed data mining algorithms, cloud comput- ing, distributed knowledge discovery, peer-to-peer systems, and parallel programming models. He is a member of the Editorial Boards of Future Gener- ation Computer Systems, the IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, the Interna-

tional Journal of Web and Grid Services, the Journal of Cloud Computing— Advances, Systems and Applications, Scalable Computing: Practice and Experience, and the International Journal of Next-Generation Computing.

PAOLO TRUNFIO was a Research Collabo- rator with the Institute of Systems and Com- puter Science, Italian National Research Council (ISI-CNR), from 2001 to 2002. In 2007, he was a Visiting Researcher with the Swedish Institute of Computer Science (SICS), Stockholm. He is currently an Associate Professor of computer engi- neering with the University of Calabria. He is in the Editorial Board of Future Generation Com- puter Systems, the IEEE TRANSACTIONS ON CLOUD

COMPUTING, and Journal of Big Data. His research interests include cloud computing, social-media analysis, service-oriented architectures, distributed knowledge discovery, and peer-to-peer systems.

VOLUME 8, 2020 47187