helpfn

IEEE INTERNET OF THINGS JOURNAL, VOL. 7, NO. 9, SEPTEMBER 2020 8133

Emotion Detection in Online Social Networks: A Multilabel Learning Approach

Xiao Zhang , Wenzhong Li , Member, IEEE, Haochao Ying, Feng Li , Member, IEEE, Siyi Tang, and Sanglu Lu, Member, IEEE

Abstract—Emotion detection in online social networks (OSNs) can benefit kinds of applications, such as personalized adver- tisement services, recommendation systems, etc. Conventionally, emotion analysis mainly focuses on the sentence level polarity prediction or single emotion label classification, however, ignor- ing the fact that emotions might coexist from users’ perspective. To this end, in this work, we address the multiple emotions detec- tion in OSNs from user-level view, and formulate this problem as a multilabel learning problem. First, we discover emotion labels correlations, social correlations, and temporal correlations from an annotated Twitter data set. Second, based on the above observa- tions, we adopt a factor graph-based emotion recognition model to incorporate emotion labels correlations, social correlations, and temporal correlations into a general framework, and detect the multiple emotions based on the multilabel learning approach. Performance evaluation demonstrates that the factor graph-based emotion detection model can outperform the existing baselines.

Index Terms—Emotion detection, factor graph, multilabel learning, online social network (OSN).

I. INTRODUCTION

W ITH the fast development of the online socialnetworks (OSNs) platforms (e.g., Twitter, Facebook, etc), expressing emotions or sharing meaningful moments with friends in OSNs has become people’s daily activities. Lots of published contents in OSNs provide a good opportunity to study users’ emotions, therefore enabling the rapid develop- ment of emotion-aware applications. For example, an activity management system or personalized advertisement system can make valuable suggestions or schedule in an emotion-sensing way. A personalized recommendation system can recommend

Manuscript received March 3, 2020; revised April 30, 2020 and May 20, 2020; accepted June 10, 2020. Date of publication June 23, 2020; date of current version September 15, 2020. This work was supported in part by the National Natural Science Foundation of China under Grant 61972196, Grant 61672278, Grant 61832008, Grant 61832005, and Grant 61702304; in part by the Fundamental Research Funds of Shandong University under Grant 2019GN105; in part by the Shandong Provincial Natural Science Foundation, China, under Grant ZR2017QF005; in part by the Industrial Internet Innovation and Development Project in 2019 of China; and in part by the Sino-German Institutes of Social Computing. (Corresponding authors: Wenzhong Li; Feng Li.)

Xiao Zhang and Feng Li are with the School of Computer Science and Technology, Shandong University, Qingdao 250100, China (e-mail: [email protected]; [email protected]).

Wenzhong Li, Siyi Tang, and Sanglu Lu are with the State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China (e-mail: [email protected]; [email protected]; [email protected]).

Haochao Ying is with the School of Public Health, Zhejiang University, Hangzhou 310027, China (e-mail: [email protected]).

Digital Object Identifier 10.1109/JIOT.2020.3004376

personalized products, movies, or songs according to individ- uals’ current emotions. When facing with public emergencies, such as earthquakes or pandemic disease, emotion detection in OSNs can be utilized to monitor public opinions, which provide support for government decision making. Therefore, emotion detection has become a critical task in OSNs and attracted more and more attentions both in academia and industry [1], [3], [43], [27].

Among the literature, existing approaches for emotion detection in OSNs could be divided into two major lines: 1) lexicon-based methods [16] and 2) machine learning- based methods [1]. The lexicon-based methods extracted emotional keywords using kinds of dictionaries, for exam- ple, the linguistic inquiry and word count (LIWC) [16]. Coviello et al. [5] utilized LIWC to measure the emotions of the published posts and detected emotional contagion in large scale OSNs. Golder and Macy [12] measured positive and negative effect of Tweets using LIWC and discovered individual-level diurnal and seasonal mood rhythms across different cultures. The machine learning-based approach [11] considered emotion detection as a prediction problem and adopted classification/regression algorithms to infer emotions.

For example, Hu et al. [15] predicted polarity sentiments of tweet texts by considering social relations. However, existing methods for emotion detection in online social media did not take the following aspects into consideration. First, emotion detection is usually inferred in sentence level, which could not reveal the user’s emotional state in a period. For instance, a user may publish multiple Tweets with various emotions at the same time. Therefore, a comprehensive analysis of emo- tions in the user level is necessary. Second, existing approaches mainly consider single emotion classification, which ignores the co-occurrence of multiple emotions in a period. Based on our observations, it is very common for a user to express multiple emotions in a period, and multiple emotions might even coexist in one sentence or one tweet. For instance, Table I illustrates some examples of tweets with multiple emotions, where more than one kind of emotions, such as “Happy” and “Surprise” might co-appear in one single tweet, which is quite different from the assumption of traditional single emotion classification. Based on the above observations, we consider the emotion detection in online social media as a multiple emotion classification problems, and technically the existing single emotion classification approaches are not suitable for this problem. Furthermore, to the best of our knowledge,

2327-4662 c© 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

8134 IEEE INTERNET OF THINGS JOURNAL, VOL. 7, NO. 9, SEPTEMBER 2020

TABLE I EXAMPLE OF TWEETS WITH MULTIPLE EMOTIONS

multiple emotions detection in OSNs has not been addressed very well.

To this end, in this article, we design a multilabel learn- ing approach for the problem of multiple emotions detection in OSNs. Specifically, as illustrated in Fig. 1, we compre- hensively extract three different user-level characteristics, the social contextual features (e.g., an individual’s emotion may be influenced by his friends on social networks), temporal features (e.g., an individual’s emotion could be correlated to his past emotion states), and textual features (the emotions revealed in his tweets), for emotion detection. Ekman’s emo- tion model [8] is first employed to express six basic categories of emotions: 1) Happy; 2) Surprise; 3) Anger; 4) Disgust; 5) Sad; and 6) Fear. Then we systematically and thoroughly study the influential factors of individuals’ emotions from an annotated Twitter data set, in which three significant correla- tions can be observed: 1) the emotion label correlation; 2) the social correlation; and 3) the temporal correlation. Definitely, the emotion label correlation denotes that several emotion pairs, such as happy and surprise are more likely to coex- ist in one instance than other emotion label pairs, such as happy and fear. The social correlation denotes that neighbor- ing users are more likely to have similar emotions in OSNs. Finally, the temporal correlation means that a user’s present emotions have high correlation with his/her emotions in the past.

After that, we propose a factor graph-based model by introducing emotion label correlation, social correlation, and temporal correlation comprehensively for the multiemotion detection problem. To be specific, the factor graph regards each variable which includes observed textual feature vari- ables and hidden label variables as a node in a graph, and the edges denote the correlations among variables, which is usually called factor function that can be utilized to represent the emotion label correlation, social correlation, and temporal correlation naturally. Then, a multilabel learning algorithm is proposed through maximizing the joint probability of the fac- tor functions. The experimental study shows that our proposed approach outperforms state-of-the-art algorithms in terms of multiple metrics.

Our contributions are summarized as follows. 1) We present the multiple emotion detection problems in

OSNs from users’ perspective which is different from traditional sentence level emotion analysis, and formu- late the multiple emotions detection as a multilabel learning problem.

2) We make several observations on a human-annotated Twitter data set and discover the correlations between emotion labels, the social relationships as well as the

Fig. 1. Example of emotion detection in OSNs.

temporal correlations, which can be utilized as features for the multiple emotions detection.

3) We propose to incorporate emotion label correlations, social correlations, and temporal correlations into a uni- fied framework based on factor graph, and solve the emotion detection problem by a multilabel learning algorithm.

4) We conduct comprehensive experiments based on the human-annotated data set, and the results show that the proposed approach can achieve better performance comparing with the state of the arts.

The remainder of this article is organized as follows. Section II introduces the emotion model and the related works on emotion analysis on texts of OSNs. Section III formal- izes the multiple emotion detection problems. Section IV introduces the Twitter data set and makes observations on the annotated data set. Section V proposes solutions to the multiple emotion detection problems, including feature extrac- tion, feature selection, and factor graph model building. Section VI reports experimental results, and demonstrates the performance of the proposed factor graph model. Section VII concludes the work.

II. RELATED WORK

A. Emotion Models

The researchers have proposed kinds of models to describe emotions or moods. A checklist with more than 20 items was used to measure negative and positive effect of kinds of aspects in the PANAS model [6]. In the Circumplex mood model [22], a 2-D circumplex was employed to represent the emotion state: the activeness dimension and the pleasure dimension. The pleasure dimension measured the degree of negative and positive feelings, and the activeness dimension measured the likelihood for a user to take action.

The emotion can also be described by the discrete cate- gory model [8], [36] based on a set of categories. One of the most popular models is Ekman’s six basic categories [8]: 1) Happy; 2) Sad; 3) Surprise; 4) Disgust; 5) Anger; and 6) Fear. Particularly, Ekman’s model is understandable and intuitive to normal users, in which multiple emotions can coex- ist. Therefore, Ekman’s emotion model was widely utilized by kinds of literature [48], [53], and it is also utilized for emotion detection in OSNs in our work.

B. Emotion Analysis on OSN Texts

Emotion analysis or sentiment analysis has drawn more and more researchers’ interests recently [28], [41], [51].

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

ZHANG et al.: EMOTION DETECTION IN ONLINE SOCIAL NETWORKS: MULTILABEL LEARNING APPROACH 8135

TABLE II COMPARISON OF KINDS OF EMOTION DETECTION WORK.

PARTICULARLY, SF. MEANS THE SOCIAL FEATURE, TMF. MEANS THE TEMPORAL FEATURE, AND TTF. MEANS THE TEXTUAL FEATURE

The goal of sentiment analysis was usually to predict the polarity (positive, negative, or neutral) of an object, which was studied in analyzing movie reviews [21], product views [14], opinion mining [38], etc. Emotion detection is usually regarded as fine-grained sentiment analysis. Emotion analysis can include more categories compared with sentiment analysis about polarity prediction. About the emotion detec- tion in the traditional human social network, Fowler et al. claimed that happiness [9], or depression [26] can spread from individual to individual based on the clinical trial. However, the amount of data collected from traditional human social network could be very limited. Thanks to the fast devel- opment of online social medias, such as Facebook, Twitter, much more data can be obtained for researchers to analyze. For instance, Moodcast [52], [35] inferred individuals’ emo- tions by considering the social influence, temporal correlation, and location information as features. Yang et al. [43] mod- eled the comment information and visual features of images jointly to further improve the performance on detecting indi- viduals’ emotions based on images. Wang et al. [40] proposed an emotion prediction model by quantifying individuals’ emo- tion influence in online image social networks such as Flicker. Zhan et al. [44] proposed a noise-aware classification frame- work on crowdsourcing emotion detection data sets in OSNs. Tang et al. [34] designed a hidden topic emotion transition model to detect both the document-level emotion and the sentence-level emotion.

Obviously, detecting emotions in OSNs, such as Facebook or Twitter, is always a hot topic [3], [39], [49]. Generally, two main methods were widely adopted for emotion detection in OSNs: 1) lexicon-based method and 2) machine learning- based method. The target of the lexicon-based method is to extract emotions from texts based on some well-known dic- tionaries, such as the LIWC dictionaries [16]. For instance, Bollen et al. [3] extracted six kinds of mood states (depres- sion, tension, vigor, anger, confusion, fatigue) from Twitter texts using the LIWC dictionary, and comparing the results with several important events. However, the lexicon-based methods are heavily dependent on the quantity and quality of words within the dictionaries. The machine learning- based methods usually detect emotions by extracting kinds of features from contents in OSNs, and then predict sen- timents or emotions utilizing various kinds of classification or regression models [15], [19], [20], [23], [39]. We com- pare kinds of emotion detection work as shown in Table II.

In detail, Vo and Collier [39] analyzed individuals’ emotions in Twitter when in earthquake situations, and then proposed kinds of emotion categories for the earthquake situations includ- ing unpleasantness, calm, anxiety, sadness, relief, and fear. Balabantaray et al. [2] analyzed emotions on the sentence level in Twitter and built a classifier to determine the emotion class of the published tweets. Hasan et al. [13] trained supervised classifiers to automatically detect and classify the emotions expressed by Twitter messages, in which avoiding high dimen- sional and sparse feature vectors. Besides training emotion classifiers, Roberts et al. [25] analyzed the linguistic style of the utilized corpus for expressing emotions. Deyu et al. [7] learned emotion distributions from social media texts by cap- turing the relations of emotions based on the Plutchik’s wheel of emotions. However, most existing literature just consider the single emotion detection, however, ignoring that multiple emotions might coexist. Different from the existing litera- ture, we consider to detect the multiple emotions in OSNs, which has not been well addressed in the past. In our previous work [50], we addressed the multiple emotion detection prob- lems in OSNs by proposing a factor graph-based model to incorporate emotion labels correlations and social correlations into a general framework. Compared with the previous work, we have rebuilt our multiple emotions detection model due to the newly observed temporal correlation. We have added the temporal correlation factor function in the updated factor graph model. We also have conducted comprehensive experiments to show the effectiveness of our proposed model.

III. PROBLEM FORMULATION

Given G = (V, E) denotes a social network graph in OSNs, such as Twitter, in which V means a user set, E ⊂ V × V represents the relationships (e.g., following) between individ- uals, respectively. For a particular individual i ∈ V , her/his posted tweets in the tth time interval are denoted as a set Sti . We generate the textual feature vector Xt from Sti . Our study aims to learn a function that maps Xt to multiple emotional labels represented by a vector Y t of the individual in the tth time interval by considering the social network information, the temporal information, and textual information

f : (

G = (V, E), Xt, Y t−1 ) → Y t (1)

where Y t denotes a multidimensional vector, whose element takes binary value “0” or “1” indicating whether the corre- sponding emotion label is chosen or not. From this view, we formalize our proposed emotion detection problem as one type of multilabel learning problem.

IV. DATA SET

A. Data Set Description

To verify the superiority of our approach, we use one popu- lar public data set, the Twitter data set [42]. To be specific, the Twitter API tool was utilized to crawl the tweet data together with the snow-ball crawling method. First, the most popular 20 users shown in the literature [18] were utilized as seed users. Then, for each seed user, all published tweets and the follow- ers list as well as the followees list can be collected. After that,

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

8136 IEEE INTERNET OF THINGS JOURNAL, VOL. 7, NO. 9, SEPTEMBER 2020

Fig. 2. Statistics of the number of emotion labels.

Fig. 3. Correlation of emotion labels.

partial lists of the users being followed by the current users can be added into the data set. The crawling period lasted for one month. All tweets posted by the individuals were up to April 2, 2010.

B. Data Preprocessing

Since the number of words on tweets is limited and much noise exist among the tweets, preprocessing the published tweets is essential. Therefore, the data set is processed as fol- lows to guarantee the quality of textual features. First, since the retweeted tweets are from other individuals which may not represent their own opinions, we remove the retweeted tweets. Second, since lots of non-English tweets might exist in the data set, we used a public tool named LangDetect [29] to remove them.

C. Data Annotation

In order to infer the individuals’ emotions, we randomly select 100 active users from the original data set for annotation, in which the active users denote these who published tweets daily. Totally, the 100 users posted 16 424 tweets. Particularly, the average of days that every individual has published tweets is 56. To be fair, we recruited 15 student volunteers to partic- ipate in the annotation process. In the beginning, participants were required to watch a prepared slide developed by the study team that described the research goals of the study and the precautions of the annotations. After simple training of the volunteers, we make sure that they understand the annotation process well and let them annotate the tweets independently. Each volunteer will be randomly assigned about 1000 tweets for annotation, and the results are used as the ground truth for model learning and performance evaluation.

Note that we employ Ekman’s discrete category model [8] to denote individuals’ emotions. Ekman’s model divides the emotions into six categories: 1) Happy; 2) Sad; 3) Surprise; 4) Anger; 5) Fear; and 6) Disgust. Therefore, we annotated the tweets using these six kinds of emotions, one kind of emotion

Fig. 4. CDF of fraction of common emotions in social correlation.

labeled with “1” means that this kind of emotion existed in the tweets, and “0” denoted the opposite.

D. Data Observation

After annotating the Twitter data set, we make some observations shown as following.

Number of Emotion Labels: We analyze the distribution of the number of emotion labels depicted in Fig. 2. As depicted in the figure, 54% of instances show one kind of emotion, while 28% and 18% of instances show two kinds and at least three kinds of emotions, respectively. That is, nearly half of instances have multiple emotions, which justifies our motivation that emotions in OSNs should be multilabeled.

Emotion Label Correlation: We count the fraction of instances that contain two coexisting emotions shown in Fig. 3, in which darker color denotes a higher frequency of coexis- tence. From this figure, we can find that some emotion pairs, such as sadness and disgust, sadness and anger, and happiness and surprise, can co-occur in the same instance frequently, however, some other pairs coexist less common. Furthermore, some pairs, such as happiness and fear and surprise and fear rarely coexist in one same instance. Therefore, the correlations between different emotion pairs are very different.

Social Correlation: In recent years, sociologists have found that emotions, such as happiness [9] and depression [26], can spread in the human social network. This indicates that a user’s emotions can be influenced by her/his social neigh- bors, which is regarded as social correlation in our work. We compare social correlation between user pairs which fol- lowed each other with randomly chosen user pairs. Similar with the literature [19], we adopt the fraction of same emotions to demonstrate the social correlation, which can be written as follows:

P ( Yi, Yj

) = 1 N

N∑ t=1

I ( Yi

t , Yj

t )

(2)

in which Y ti denotes the individual i’s label set at time interval t, I(·) denotes the binary function, which has value 1 if the label set Y ti and Y

t j have at least one same kind of emotion

and N means the number of total instances. In addition, we select 20 user pairs which followed each other randomly. Then the same number of nonfriend user pairs is selected from the annotated data set randomly. As can be seen in Fig. 4, the fraction of same emotions of 60% friends pairs is larger than 0.6, however, only 15% nonfriends pairs obtain fraction larger than 0.6. Moreover, the mean value of the fraction of com- mon emotions among friends pairs is 0.52, however, the mean

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

ZHANG et al.: EMOTION DETECTION IN ONLINE SOCIAL NETWORKS: MULTILABEL LEARNING APPROACH 8137

Fig. 5. Jaccard similarity between emotion states of present day t and the emotion states of the day t − τ.

value of the fraction among the nonfriends pairs is 0.31. This observation indicates that if two users are friends, these two individuals are more likely to show similar emotions in Twitter at same time interval.

Temporal Correlation: Researchers have verified that an individual’s present mental status strongly depends on the previous mental state [33]. In order to capture the temporal correlation, we calculate the Jaccard correlation between emo- tion states of present day t and the emotion states of the day t−τ utilizing (3), in which Y ti indicates user i’s emotion states of day t. The higher Jaccard coefficient is, the more similar two label sets are

Sim ( Yi

t , Yi

t−τ ) = ∣∣Yit ∩ Yit−τ

∣∣ ∣∣Yit ∪ Yit−τ

∣∣. (3)

We make statistics on all users’ temporal correlations ranging τ from 1 to 11 and calculate the mean value under each τ. The result is depicted in Fig. 5. We observe that the emotion states of the present day t have the highest correlation with the emotion states of the day t − 1 as well as the smallest standard deviation. With τ becomes larger, the similarity tends to become smaller. Especially when k is larger than 7, the similarity declines sharply. Since the correlation is the highest when τ = 1, in our experiment, we only utilize the users’ previous emotion states of day t−1 to infer the users’ emotion at day t.

In summary, our observations demonstrates several signifi- cant correlations in the data set.

1) Emotion Labels Correlations: Several emotion label pairs are more likely to coexist than other emotion label pairs in one same instance.

2) Social Correlation: User pairs who follow each other in OSNs are more likely to show same kind of emotions.

3) Temporal Correlation: An individual’s emotional states at day t is highly correlated with his/her emotion state at day t − 1.

In the following section, we will take such correlations into account to build a multilabel learning model for the multiple emotion detection problems in OSNs.

V. SOLUTION FRAMEWORK

Based on the above analysis, we fuse the emotion labels correlation, social correlation, and temporal correlation into the unified factor graph model [17], and propose a multilabel learning algorithm for multiple emotions detection. The factor graph models consider every variable as one node in a graph. The edges denote the correlations among different variables,

which are called factor functions. Therefore, the factor graph can model the emotion labels correlation, social correlation, and temporal correlation as factor functions naturally. In addi- tion, since the dimension of textual features might be larger than the number of total instances, we utilized feature selec- tion approaches to reduce the complexities of the multilabel emotion detection model. The details on the model are shown as follows.

A. Textual Features

First, the Unigram model [30] is adopted to extract tex- tual features from the annotated data set, which has been proven to be effective in literature [11]. Then, similar to lit- erature [15], we reserved stemming or stop words because these words might include emotional information. Besides, we denote every tweet as a textual feature vector with multiple dimensions based on the Unigram model. Normally the dimen- sions of the extracted feature vectors are usually much larger than the number of the total instances. Therefore, we design a feature selection approach to reduce the computational complexities.

B. Feature Selection

The goal of feature selection is to find a subset of features X∗ ⊆ X, in which X∗ can describe the data set as well as the original feature space X does. Technically, feature selection provides a good way to reduce the dimensions, especially for the high-dimensional data.

As for the multilabel emotion detection problem, the problem transformation approach as shown in [31] is adopted to select significant features. The following are the steps.

1) First, binary relevance (BR) [37] as one popular trans- formation method is utilized to transform the multilabel data set to K single label data sets, in which K denotes the number of emotion labels.

2) Then, in each single label data set, we apply ReliefF [32] to measure the importance of the features. The ReliefF- based feature selection method outputs a value w for each feature, ranging from −1 to 1. The larger w means the feature is more significant.

3) For one particular feature in the all K single label data sets, summing up the K importance outputs as the significant measure of this feature.

4) Finally, sorting all the features and selecting the top K significant features according to their significance measurements.

C. Factor Graph Model

By taking the before-mentioned kinds of correlations into account, we propose a factor graph-based emotion detection model to infer individuals’ emotions as described in Fig. 6.

Particularly, a feature vector Xti includes N feature attributes after feature selection, in which Xti denotes the ith individual’s posted tweets in the tth time interval. The emotion labels at the tth time interval can be denoted as yti = (yti1, . . . , ytiK), in which K represents the number of different emotions. To consider the correlations between features and emotion labels,

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

8138 IEEE INTERNET OF THINGS JOURNAL, VOL. 7, NO. 9, SEPTEMBER 2020

Fig. 6. Factor graph model for multiple emotions detection.

emotion labels correlations (the correlations among an individ- ual’s different emotions), social correlations (the correlations between two different individuals’ emotions), as well as the temporal correlations, four kinds of factor functions are illus- trated in the proposed factor graph-based emotion detection model.

1) Textual Feature-Label Factor: f (Xti, y t ik) denotes the cor-

relation between the individual i’s kth emotion label ytik at time t and the corresponding feature vector Xti . The feature-label factor measures the correlation between emotion labels and feature vector. The red rectangles represent the feature-label factor functions as shown in Fig. 6.

Similar to the traditional multilabel learning approaches, the feature vector Xti is replicated by K copies (denoted as {Xtik}Kk=1), such that the copy Xtik is associated with the emo- tion label ytik as shown in Fig. 6. Then, the feature-label factor function is defined as

f ( Xtik, y

t ik

) = 1 Z1

exp

{ N∑

n=1 αikn�

( Xtikn, y

t ik

) }

(4)

in which �(Xtikn, y t ik) denotes a binary indicator function. For

instance, �(Xtikn = “true”, ytik = “1”) denotes that if the individual i has the kth emotion and the nth attribute in feature vector Xtikn is true, then the value of the function is 1, otherwise the value is 0. αikn means the weight of the function. αikn describes how strong the correlation between Xikn and y

t ik is.

These are also the parameters which need to be learned, and Z1 is a normalization term.

2) Label Correlation Factor: Given w(ytik, y t id) denotes the

correlation between individual i’s kth emotion label ytik and individual i’s dth emotion label ytid. The yellow rectangles denote the label correlation factor functions as depicted in Fig. 6. Since every instance in the data set has multiple emotions, if taking all the possible emotion labels correla- tions among any two labels into account, the exact inference in the factor graph-based emotion detection model might be very difficult. Therefore, utilizing the observations mentioned before, we design Algorithm 1 to elicit the strong correlation

Algorithm 1 Construct Label Correlation Tree Input: Label matirx M, Number of labels K Output: Label correlation graph G = (V, E)

1: Count the times tij that label pair < li, lj > coexist in each row of M, T ={< li, lj >, tij}Ki,j=1&i =j

2: Sort T according to the times tij 3: for each label pair < li, lj > in T do 4: if |V| < K then 5: if li ∈ V &lj ∈ V then 6: continue; 7: else 8: V = V ∪{li, lj}; 9: E = E ∪ (li, lj);

10: end if 11: else 12: break; 13: end if 14: end for

label pairs, which avoids the cyclic structure in the emotion detection model.

First, the frequencies that two emotion labels coexist in one same instance are sorted in a descending order. Then the emo- tion label pairs are selected to construct the label correlation tree according to the order. The set of individual i’s emotion labels which have correlation with the label ytik is represented as W(ytik) on the tree. Similarly, we define the factor function w(ytik, y

t id) for any y

t id ∈ W(ytik) as

w ( ytik, y

t id

) = 1 Z2

exp { βikd�

( ytik, y

t id

)} (5)

in which βkd denotes the weight and quantifies the influence degree between two different emotion labels. Then we define the factor function w(ytik, W(y

t ik)) which can describe the

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

ZHANG et al.: EMOTION DETECTION IN ONLINE SOCIAL NETWORKS: MULTILABEL LEARNING APPROACH 8139

collective influence between the set ytik and the set W(y t ik) as

w ( ytik, W

( ytik

)) = 1 Z3

exp

⎧ ⎨ ⎩

∑

ytid∈W(yik qit) βikd�

( ytik, y

t id

) ⎫ ⎬ ⎭. (6)

3) Social Correlation Factor: Given h(ytik, y t jk) denote the

correlation between individual i’s kth label ytik and individ- ual j’s kth label ytjk. To consider the social correlation in the unified factor graph model, similar with the label correlation factor function, we define h(ytik, y

t jk) shown as follows. The

green rectangles denote the social correlation factor functions as shown in Fig. 6

h (

ytik, y t jk

) = 1

Z4 exp

{ δijk�

( ytik, y

t jk

)} (7)

h ( ytik, H

( ytik

)) = 1 Z5

exp

⎧ ⎪⎨ ⎪⎩

∑

ytjk∈H(ytik) δijk�

( ytik, y

t jk

) ⎫ ⎪⎬ ⎪⎭

(8)

in which H(ytik) means the set of individual i’s friends’ kth emotion labels at time t.

4) Temporal Correlation Factor: Let ψ(ytik, y t−1 ik ) denote

the correlation between individual i’s kth label ytik at time t and individual i’s kth label yt−1ik at time t − 1. The definition is shown in 9. The blue rectangles in Fig. 6 represent the temporal correlation factors

ψ (

ytik, y t−1 ik

) = 1

Z5 exp

{ χit�

( ytik, y

t−1 ik

)} . (9)

Based on the factor functions described before, we define the objective function as

P(Yi|Xi,θi) = T∏

t=1

K∏ k=1

f ( ytik, X

t ik

) w

( ytik, W

( ytik

))

h ( ytik, H

( ytik

)) ψ

( ytik, y

t−1 ik

) . (10)

D. Learning Algorithm

We obtain the optimal parameter configuration θ = {α,β,δ,χ} by minimizing the following negative log- likelihood objective function together with a L2-regularization penalty to prevent overfitting

L(θi) =− log P(Yi|Xi,θi)+ λ

2

N∑ n=1

K∑ k=1

α 2

ikn

+ λ 2

K∑ k=1

∑

d∈W(ytik) βikd

2 + λ 2

K∑ k=1

∑

j∈H(ytik) δ

2 ijk

+ λ 2

T∑ t=1

χit 2 . (11)

We adopt the gradient descent approach to learn the parame- ters θ, and we calculate the gradients for our emotion detection model as ∂L(θi)

∂αikn = Eθi

[ �

( Xtikn, y

t ik

)] − EDi [ �

( Xtikn, y

t ik

)] +λαikn (12)

Algorithm 2 Learning Algorithm for Factor Graph Model Input: Learning rate η Output: Model parameters θi

1: Initialize θi ← 0 2: repeat 3: Calculate EDi [�(X

t ikn, y

t ik)],EDi [�(y

t ik, y

t id)],

EDi [�(y t ik, y

t jk)], EDi [�(y

t ik, y

t−1 ik )] using the training

dataset 4: Calculate Eθi [�(X

t ikn, y

t ik)],Eθi [�(y

t ik, y

t id)],

Eθi [�(y t ik, y

t jk)], Eθi [�(y

t ik, y

t−1 ik )] using BP algorithm

5: Calculate the gradients ∂L(θi) ∂θi

according to Eqs.(12), (13), (14) and (15)

6: Update parameter θi as θi = θi +η∂L(θi)∂θi 7: until Convergence

∂L(θi)

∂βikd = Eθi

[ �

( ytik, y

t id

)] − EDi [ �

( ytik, y

t id

)] +λβikd (13) ∂L(θi)

∂δijk = Eθi

[ �

( ytik, y

t jk

)] − EDi

[ �

( ytik, y

t jk

)] +λδijk (14)

∂L(θi)

∂χit = Eθi

[ �

( ytik, y

t−1 ik

)] − EDi

[ �

( ytik, y

t−1 ik

)] +λχit

(15)

in which Eθ [ ·] denotes the expectation of feature values with respect to the model parameters θ, ED[ ·] denotes the average by counting the given pattern over the given training data set. The learning algorithm is depicted in Algorithm 2, in which we adopt the belief propagation (BP) algorithm [10] to calculate the expectation value Eθ [ · ].

Based on the learned parameter θ, we can infer individu- als’ multiple emotions utilizing the principle of Maximum a Posteriori (MAP), for example, by maximizing the likelihood based on the parameters θ, we can find the emotion labels as

arg max P(Y = y|X,θ). (16) We utilize the BP approach to calculate the marginal proba- bilities. Therefore, the emotions with the largest probabilities can be identified.

VI. PERFORMANCE EVALUATION

A. Experimental Setup

To validate the effectiveness and efficiency of the proposed emotion detection model, we conduct extensive experiments in the Twitter data set. More specifically, for each individual, we randomly choose 70% data as training set, while the remain- ing as test set for performance evaluation. Note that we select significant features on each user’s training data set. Otherwise, we will also discuss the influence of several important parame- ters, including the number of selected features and the training data set size in Section VI-D.

1) Baselines: We compare our proposed approach with several widely used classification baselines as follows.

1) BR, which is also called the problem transformation method, transforms the multilabel data set to k single label data sets. The BR method applies some basic algo- rithms on each data set separately, such as Decision

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

8140 IEEE INTERNET OF THINGS JOURNAL, VOL. 7, NO. 9, SEPTEMBER 2020

TABLE III MEAN VALUE OF EVALUATION RESULTS OF ALL USERS

Tree, SVM, Logistic Regression, etc. Then combining the results to form the predicted multiple labels. In our experiments, we utilize the Decision Tree as the basic classifier for BR due to the fact that Decision Tree per- forms better than SVM and Logistic Regression in our learning task.

2) Label combination (LC) [37] method considers each LC as a single class in a multiclass classification problem.

3) Backpropagation neural network (BPNN) [45] is a neu- ral network that have multiple outputs correspond to multiple labels.

4) Probabilistic classifier chains (PCC) [4] considers out- puts as new inputs for the next classifiers in the chain when training models and explores all possible paths applying models in testing.

5) ML-KNN [46] is from the traditional KNN algorithm, which utilizes K nearest neighbors in the training set to predict the instances in the test data set.

Particularly and fairly, we implement the baselines, BR, LC, BPNN, PCC, based on the MEKA toolbox [24], while ML- KNN is from the authors’ public code.

B. Performance Metrics

To evaluate the performance of each approach, we employ several widely used metrics for multilabel learning [47], including accuracy, precision, recall, and F1-score, as follows.

1) Accuracy: The number of correctly predicted labels divided by the total number of labels in one instance. Then the overall accuracy denotes the mean on accuracy of all instances

Accuracy = 1 p

p∑ i=1

|Yi ∩ Pi| |Yi ∪ Pi|

. (17)

2) Precision: The number of correctly predicted labels divided by the number of predicted labels in one instance. Then the overall precision denotes the mean on precision of all instances

Precision = 1 p

p∑ i=1

|Yi ∩ Pi| |Pi|

. (18)

3) Recall: The number of correctly predicted labels divided by the number of true labels in one instance. Then the overall recall denotes the mean on recall of all instances

Recall = 1 p

p∑ i=1

|Yi ∩ Pi| |Yi|

. (19)

Fig. 7. CDFs on accuracy of different models.

Fig. 8. CDFs on precision of different models.

Fig. 9. CDFs on recall of different models.

4) F1 Score: Calculated as the weighted harmonic mean between recall and precision

F1 = 2 ∗ Recall ∗ Precision Recall + Precision (20)

in which p represents the number of total instances, Yi denotes the ground-truth emotion label set, and Pi denotes the predicted emotion label set.

C. Experiment Results

Table III demonstrates the performance of our factor graph- based emotion detection model together with baselines under each metric. On average, compared with BR, the proposed emotion detection model co achieve +10.4% accuracy, +13.2% precision, +10.6% recall , and +5.7% F1-score improvement. BPNN achieves the best performance among the baselines. Compared with BPNN, the proposed emotion detec- tion model could achieve +6.2% accuracy, +10.9% precision, +9.7% recall, and +5.0% F1-score improvement. By incor- porating more information into the factor graph model, such as emotion label correlation, social correlation, and temporal correlation, the proposed model can perform the best.

The CDF of accuracy, precision, recall, and F1-score are depicted in Figs. 7–10, respectively. As shown in Fig. 7, 70% factor graph-based emotion detection models could achieve accuracy larger than 50%, however, 35% BR models, 60% BPNN models, 30% PCC models, 30% LC models, and 50% ML-KNN models achieve accuracy larger than 50%. Similarly

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

ZHANG et al.: EMOTION DETECTION IN ONLINE SOCIAL NETWORKS: MULTILABEL LEARNING APPROACH 8141

Fig. 10. CDFs on F1-score of different models.

Fig. 11. Mean accuracy under different training data set size.

Fig. 12. Mean F1-score under different training data set size.

Fig. 13. Mean accuracy under different number of selected features.

in Fig. 10, 90% factor graph-based emotion detection models obtain F1-score larger than 50%, however, 75% BR models, 70% BPNN models, 60% PCC models, 60% LC models, and 80% ML-KNN models have F1-score larger than 50%. This justifies the effectiveness of our factor graph-based emotion detection model.

D. Parameter Analysis

In this section, we study the impact of two important param- eters, the size of the training data set and the number of selected features , respectively. First, the impact of the size of the training data set under Accuracy and F1-score is shown in Figs. 11 and 12, respectively. With the increase in the size of the training data set from 40% to 70%, both accu- racy and F1-score become larger. However, when the size of the training data set continues to increase to 80%, some base- line approaches still increase, but the proposed factor approach decrease slightly, this can be caused by many reasons, and

Fig. 14. Mean F1-score under different number of selected features.

probably the performance degradation is due to that the model is overfitted and lacks generalization. Then, the influence of the number of selected features is shown in Figs. 13 and 14. As shown in the result, when the number of selected features increases from 4 to 6, the accuracy and the F1-score become larger, since more information are taken into account by the classification model. However, when the number increases from 6 to 10, both accuracy and F1-score tend to become smaller. This indicates that when the number of instances is fixed, the increment of the number of the selected features could lead to overfitting of the model.

VII. CONCLUSION

In this work, the multiple emotion detection problems in OSNs was studied based on multilabel learning. On the basis of the observations on an annotated Tweet data set, we dis- covered the emotion label correlation and social correlation as well as the temporal correlation in OSNs. Furthermore, a factor graph model is introduced to incorporate the above correlations into a general framework for multiple emotions detection. Then we proposed a multilabel learning algorithm to address the problem. The conducted experiments show that our proposed approach outperforms other existing baselines.

The study of emotion detection in OSNs achieves several cheerful results by considering contextual information, such as temporal information, social information, etc. The proposed model can be flexible to work in the circumstance when some factor functions were absent. However, we shall also mention some limits of the work and the possible future directions. Our proposed model is only tested on a small scale data set. As we all know, annotating data is very time consuming. Due to the limit of manpower, we can only annotate a limited number of users to form the data set. The annotating process also lacks retrospective check. In the next step, we may annotate larger scale higher quality data set or adopt semi-supervised learning approaches to detect emotions in OSNs. In addition, as emotion is time-series data, we may apply the advanced technique to detect users’ emotional states, such as LSTM, attention-based neural network, etc.

REFERENCES

[1] C. O. Alm, D. Roth, and R. Sproat, “Emotions from text: Machine learning for text-based emotion prediction,” in Proc. Conf. Human Lang. Technol. Empir. Methods Nat. Lang. Process., 2005, pp. 579–586.

[2] R. C. Balabantaray, M. Mohammad, and N. Sharma, “Multi-class Twitter emotion classification: A new approach,” Int. J. Appl. Inf. Syst., vol. 4, no. 1, pp. 48–53, 2012.

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

8142 IEEE INTERNET OF THINGS JOURNAL, VOL. 7, NO. 9, SEPTEMBER 2020

[3] J. Bollen, H. Mao, and A. Pepe, “Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena,” in Proc. 5th Int. AAAI Conf. Weblogs Soc. Media (ICWSM), vol. 11, 2011, pp. 450–453.

[4] W. Cheng, E. Hüllermeier, and K. J. Dembczynski, “Bayes optimal multilabel classification via probabilistic classifier chains,” in Proc. 27th Int. Conf. Mach. Learn. (ICML), 2010, pp. 279–286.

[5] L. Coviello et al., “Detecting emotional contagion in massive social networks,” PLoS ONE, vol. 9, no. 3, 2014, Art. no. e90315.

[6] J. R. Crawford and J. D. Henry, “The positive and negative affect sched- ule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample,” Brit. J. Clin. Psychol., vol. 43, no. 3, pp. 245–265, 2004.

[7] Z. Deyu, X. Zhang, Y. Zhou, Q. Zhao, and X. Geng, “Emotion distri- bution learning from texts,” in Proc. Conf. Empir. Methods Nat. Lang. Process., 2016, pp. 638–647.

[8] P. Ekman et al., “Universals and cultural differences in the judgments of facial expressions of emotion,” J. Pers. Soc. Psychol., vol. 53, no. 4, pp. 712–717, 1987.

[9] J. H. Fowler and N. A. Christakis, “Dynamic spread of happiness in a large social network: Longitudinal analysis over 20 years in the framingham heart study,” Brit. Med. J., vol. 337, Dec. 2008, Art. no. a2338.

[10] B. J. Frey and D. J. MacKay, “A revolution: Belief propagation in graphs with cycles,” in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 1998, pp. 479–485.

[11] A. Go, R. Bhayani, and L. Huang, “Twitter sentiment classification using distant supervision,” Dept. Comput. Sci., Stanford Univ., Stanford, CA, USA, Rep. CS224N, 2009.

[12] S. A. Golder and M. W. Macy, “Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures,” Science, vol. 333, no. 6051, pp. 1878–1881, 2011.

[13] M. Hasan, E. Rundensteiner, and E. Agu, “EMOTEX: Detecting emo- tions in twitter messages,” in Proc. Int. Conf. Bigdata Cyber Security, 2014, pp. 27–31.

[14] M. Hu and B. Lium, “Mining and summarizing customer reviews,” in Proc. 10th ACM SIGKDD Int. Conf. Knowl. Discover. Data Min., 2004, pp. 168–177.

[15] X. Hu, L. Tang, J. Tang, and H. Liu, “Exploiting social relations for sentiment analysis in microblogging,” in Proc. 6th ACM Int. Conf. Web Search Data Min., 2013, pp. 537–546.

[16] A. D. Kramer, “An unobtrusive behavioral model of ‘gross national happiness”’ in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2010, pp. 287–290.

[17] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498–519, Feb. 2001.

[18] H. Kwak, C. Lee, H. Park, and S. Moon, “What is twitter, a social network or a news media?” in Proc. 19th Int. Conf. World Wide Web, 2010, pp. 591–600.

[19] S. Li, L. Huang, R. Wang, and G. Zhou, “Sentence-level emotion clas- sification with label and context dependence,” in Proc. Annu. Meeting Assoc. Comput. Linguist. (ACL), 2013, pp. 1045–1053.

[20] S. M. Mohammad and F. Bravo-Marquez, “WASSA-2017 shared task on emotion intensity,” 2017. [Online]. Available: arXiv:1708.03700.

[21] B. Pang and L. Lee, “A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts,” in Proc. 42nd Annu. Meeting Assoc. Comput. Linguist., 2004, p. 271.

[22] J. Posner, J. A. Russell, and B. S. Peterson, “The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology,” Develop. Psychopathol., vol. 17, no. 03, pp. 715–734, 2005.

[23] M. Purver and S. Battersby, “Experimenting with distant supervision for emotion classification,” in Proc. 13th Conf. Eur. Chapter Assoc. Comput. Linguist., 2012, pp. 482–491.

[24] J. Read, P. Reutemann, B. Pfahringer, and G. Holmes, “MEKA: A multi- label/multi-target extension to Weka,” J. Mach. Learn. Res., vol. 17, no. 21, pp. 1–5, 2016.

[25] K. Roberts, M. A. Roach, J. Johnson, J. Guthrie, and S. M. Harabagiu, “Empatweet: Annotating and detecting emotions on twitter,” in Proc. Int. Conf. Lang. Resources Eval. (LREC), vol. 12, 2012, pp. 3806–3813.

[26] J. N. Rosenquist, J. H. Fowler, and N. A. Christakis, “Social network determinants of depression,” Mol. Psy., vol. 16, no. 3, pp. 273–281, 2011.

[27] K. Saha, L. Chan, K. De Barbaro, G. D. Abowd, and M. De Choudhury, “Inferring mood instability on social media by leveraging ecologi- cal momentary assessments,” in Proc. ACM Interact. Mobile Wearable Ubiquitous Technol., vol. 1, no. 3, pp. 1–27, 2017.

[28] H. Sheng et al., “Mining hard samples globally and efficiently for person re-identification,” IEEE Internet Things J., early access, Mar. 13, 2020, doi: 10.1109/JIOT.2020.2980549.

[29] N. Shuyo. (2010). Language Detection Library for Java. [Online]. Available: https://code.google.com/archive/p/language-detection/

[30] V. Siivola and B. L. Pellom, “Growing an n-gram language model,” in Proc. 9th Eur. Conf. Speech Commun. Technol., 2005, pp. 1309–1312.

[31] N. Spolaôr, E. A. Cherman, M. C. Monard, and H. D. Lee, “A com- parison of multi-label feature selection methods using the problem transformation approach,” Electron. Notes Theor. Comput. Sci., vol. 292, pp. 135–151, Mar. 2013.

[32] N. Spolaôr, E. A. Cherman, M. C. Monard, and H. D. Lee, “ReliefF for multi-label feature selection,” in Proc. Brazil. Conf. Intell. Syst. (BRACIS), Fortaleza, Brazil, 2013, pp. 6–11.

[33] Y. Suhara, Y. Xu, and A. Pentland, “DeepMood: Forecasting depressed mood based on self-reported histories via recurrent neural networks,” in Proc. 26th Int. Conf. World Wide Web, 2017, pp. 715–724.

[34] D. Tang, Z. Zhang, Y. He, C. Lin, and D. Zhou, “Hidden topic–emotion transition model for multi-level social emotion detection,” Knowl. Based Syst., vol. 164, pp. 426–435, Jan. 2019.

[35] J. Tang et al., “Quantitative study of individual emotional states in social networks,” IEEE Trans. Affective Comput., vol. 3, no. 2, pp. 132–144, Apr.–Jun. 2012.

[36] S. S. Tomkins, Affect, Imagery, Consciousness: Vol. I. The Positive Affects. New York, NY, USA: Springer, 1962.

[37] G. Tsoumakas, I. Katakis, and I. Vlahavas, “Mining multi-label data,” in Data Mining and Knowledge Discovery Handbook. Boston, MA, USA: Springer, 2009, pp. 667–685.

[38] G. Vinodhini and R. Chandrasekaran, “Sentiment analysis and opinion mining: A survey,” Int. J., vol. 2, no. 6, pp. 282–292, 2012.

[39] B. Vo and N. Collier, “Twitter emotion analysis in earthquake situations,” Int. J. Comput. Linguist. Appl., vol. 4, no. 1, pp. 159–173, 2013.

[40] X. Wang, J. Jia, J. Tang, B. Wu, L. Cai, and L. Xie, “Modeling emotion influence in image social networks,” IEEE Trans. Affective Comput., vol. 6, no. 3, pp. 286–297, Jul.–Sep. 2015.

[41] Y. Xiao, Y. Jia, C. Liu, X. Cheng, J. Yu, and W. Lv, “Edge computing security: State of the art and challenges,” Proc. IEEE, vol. 107, no. 8, pp. 1608–1631, Aug. 2019.

[42] T. Xu, Y. Chen, L. Jiao, B. Y. Zhao, P. Hui, and X. Fu, “Scaling microblogging services with divergent traffic demands,” in Proc. 12th Int. Middlew. Conf., 2011, pp. 20–40.

[43] Y. Yang et al., “How do your friends on social media disclose your emotions?” in Proc. AAAI Conf. Artif. Intell. (AAAI), vol. 14, 2014, pp. 1–7.

[44] X. Zhan, Y. Wang, Y. Rao, and Q. Li, “Learning from multi-annotator data: A noise-aware classification framework,” ACM Trans. Inf. Syst., vol. 37, no. 2, pp. 1–28, 2019.

[45] M.-L. Zhang and Z.-H. Zhou, “Multilabel neural networks with appli- cations to functional genomics and text categorization,” IEEE Trans. Knowl. Data Eng., vol. 18, no. 10, pp. 1338–1351, Oct. 2006.

[46] M.-L. Zhang and Z.-H. Zhou, “ML-KNN: A lazy learning approach to multi-label learning,” Pattern Recognit., vol. 40, no. 7, pp. 2038–2048, 2007.

[47] M.-L. Zhang and Z.-H. Zhou, “A review on multi-label learning algo- rithms,” IEEE Trans. Knowl. Data Eng., vol. 26, no. 8, pp. 1819–1837, Aug. 2014.

[48] X. Zhang, W. Li, X. Chen, and S. Lu, “MoodExplorer: Towards com- pound emotion detection via smartphone sensing,” ACM Interact. Mobile Wearable Ubiquitous Technol., vol. 1, no. 4, p. 176, 2018.

[49] X. Zhang et al., “Predicting happiness state based on emotion repre- sentative mining in online social networks,” in Proc. Pac.–Asia Conf. Knowl. Discover. Data Min., 2017, pp. 381–394.

[50] X. Zhang, W. Li, and S. Lu, “Emotion detection in online social network based on multi-label learning,” in Proc. Int. Conf. Database Syst. Adv. Appl., 2017, pp. 659–674.

[51] X. Zhang, F. Zhuang, W. Li, H. Ying, H. Xiong, and S. Lu, “Inferring mood instability via smartphone sensing: A multi-view learning approach,” in Proc. 27th ACM Int. Conf. Multimedia, 2019, pp. 1401–1409.

[52] Y. Zhang, J. Tang, J. Sun, Y. Chen, and J. Rao, “MoodCast: Emotion prediction via dynamic continuous factor graph model,” in Proc. IEEE 10th Int. Conf. Data Min., Sydney, NSW, Australia, 2010, pp. 1193–1198.

[53] S. Zhao, H. Yao, and X. Jiang, “Predicting continuous probability distri- bution of image emotions in valence-arousal space,” in Proc. 23rd ACM Int. Conf. Multimedia, 2015, pp. 879–882.

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.

ZHANG et al.: EMOTION DETECTION IN ONLINE SOCIAL NETWORKS: MULTILABEL LEARNING APPROACH 8143

Xiao Zhang received the B.S. degree from Central South University, Changsha, China, in 2013, and the Ph.D. degree from Nanjing University, Nanjing, China, in 2019.

He is currently an Assistant Professor with the School of Computer Science and Technology, Shandong University, Jinan, China. His research interests include data mining, pervasive computing, and social networks.

Wenzhong Li (Member, IEEE) received the B.S. and Ph.D. degrees in computer science from Nanjing University, Nanjing, China, in 2002 and 2007, respectively.

He was an Alexander von Humboldt Scholar Fellow with the University of Göettingen, Göettingen, Germany. He is currently a Professor with the Department of Computer Science, Nanjing University. He has published over 80 peer-review papers at international conferences and journals, which include INFOCOM, UBICOMP, IJCAI,

ICDCS, IWQoS, IEEE Communications Magazine, the IEEE/ACM TRANSACTIONS ON NETWORKING, the IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, the IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, and Computer Networks. His research interests include distributed computing, data mining, mobile cloud computing, wireless networks, pervasive computing, and social networks.

Dr. Li was also the winner of the Best Paper Award of ICC 2009 and APNet 2018. He served as Program co-chair of MobiArch 2013 and the Registration Chair of ICNP 2013. He was the TPC member of several international conferences and the reviewer of many journals. He is the Principle Investigator of three fundings from NSFC, and the co-principle investigator of a China–Europe international research staff exchange program. He is a member of ACM and China Computer Federation.

Haochao Ying received the B.S. degree in computer science and technology from Zhejiang University of Technology, Hangzhou, China, in 2014, and the Ph.D. degree from the College of Computer Science, Zhejiang University, Hangzhou, in 2019.

He is currently an Assistant Professor with the School of Public Health, Zhejiang University. He has authored some papers at prestigious international conferences and journals, such as World Wide Web Journal, IJCAI, CVPR, WSDM, and PAKDD. His research interests include data mining for healthcare

and personalized recommender system.

Feng Li (Member, IEEE) received the B.S. degree in computer science from Shandong Normal University, Jinan, China, in 2007, the M.S. degree in computer science Shandong University, Jinan, in 2010, and the Ph.D. degree in computer science from Nanyang Technological University, Singapore, in 2015.

From 2014 to 2015, he worked as a Research Fellow with the National University of Singapore, Singapore. After that, he joined the School of Computer Science and Technology, Shandong

University, where he is currently an Associate Professor. His research interests include distributed algorithms and systems, wireless networking, mobile computing, and Internet of Things.

Siyi Tang is currently pursuing the undergraduation degree with the Department of Computer Science, Nanjing University, Nanjing, China.

Her research interests include data mining and social networks.

Sanglu Lu (Member, IEEE) received the B.S., M.S., and Ph.D. degrees in computer science from Nanjing University, Nanjing, China, in 1992, 1995, and 1997, respectively.

She is currently a Professor with the Department of Computer Science and Technology and the Deputy Director of State Key Laboratory for Novel Software Technology, Nanjing University. Her research interests include distributed computing, per- vasive computing, and wireless networks. She has published more than 100 papers in referred journals

and conferences in the above areas. Prof. Lu is a member of ACM.

Authorized licensed use limited to: University of the Cumberlands. Downloaded on September 25,2021 at 02:15:09 UTC from IEEE Xplore. Restrictions apply.