help

Research on the Key Technology of Chinese Text Sentiment Analysis

X ingtong Ge and Xiaofang Jin * School ofInformat ion Engineer ing

Communication University ofChina Beijing, China

gxt_hannah@126 .com, [email protected]

Abstra ct- In th e era of bi g data , text senti ment an alysis is of gre at significance to th e a na lysis of public opinion. In genera l, th er e a re two broad a ppr oaches on senti ment a na lysis, lexicon-b ased an d machine learning-ba sed method. In fact , senti ment analysis belon gs to the cla ssification tec hni que as well. Th er efore, thi s pap er a lso st udie d t he method ba sed on deep Iearnlng, T his pap er impl emented thre e approaches a nd compared t he performanc es of differ ent cla ssification effect s. The conce pt of W or dlvec was also introduced to the ma chine learning-based method. Th e wor d vect or izati on method wa s used to ext ra ct t he cor pus features a n d r edu ce the dimension through the Prtnclpal Component A nalysis (PCA) a lgor it hm. The fully conn ect ed neural network wa s selecte d in deep-learning-ba sed method. This pap er used k er ns library to build neural network fr amework. By comparing th e three methods, it was conclud ed th at the machine learning method wa s the best. T he cor r ect ra te was 85.60%.

K eyw ords- sentiment analysis;machine l earning; IVord2vec; l exicon

I. I NTROD UCTION

With the development of internet technologies, a large number of users tend to submit their positive or negative remarks and commentary analysis with emotional tendencies to the internet. The huge amount of their comments data can be classified into certain categories to measure the tendency of the public attitude which can be used to help us to understand the public's perception of a product or an event. Accordingly, the sentiment analysis is proposed. Sentiment analysis is a process of identifying sentiment for some texts written in a natural language with subj ective colors, such as movie reviews, user experiences for using product comments, and so on [1]. These subjective evaluations will affect the judgments of public, so the sentiment analysis technologies have obtained sustained attention. In general, there are two main methods of sentiment analysis, lexicon-based and machine learning-based method [2]. Md. Al [3] used word vectors method in sentiment analysis of Benga li comments. RJose [4] used an approach that combined machine learning classifiers with lexicon-based classifiers to classify sentiment. 1. Li [5] considered the relationship between emotion words when analyze the sentiment. W.Zongyue [6] proposed a method which can make sentiment analysis for special field commentary with less external knowledge. In fact, sentiment analysis can be transformed into classification

978-1-5386 -65 65-7/1 8/$3 1.00 (f)2018 IEEE

395

Bo M iao, Chenming Liu and Xinyi Wu Academy ofBroadcast ing Science

Radio. Film & Television Beijing . China

[email protected] , [email protected], [email protected]

problem, therefore, deep learning-based approach [7] can also be used in sentiment analysis. Previous studies rarely used deep learning method, so this paper added this method and proposed the implementation process of the three methods. This paper compared the classification effects. The comparison result can be used as a reference for practical analysis.

II. RELATED W ORK

A . Lexicon -based method Lexicon-based method is to formulate a series of sentiment

dictionaries and rules, to split and parse paragraphs, to find out the emotional words, negative words and degree adverbs in the document, and finally calculate emotional values to determine the emotional tendencies of the text [8].

B . Ma chine learning-based meth od Machine learning-based method is to use statistical learning

from marked corpus to extract features which can effectively express the type characteristics of comments and form the classification model by using those features [9]. In order to translate the corpus into computer understandable form, the corpus needs to be transferred into corresponding vector representat ion [10]. This paper introduced an open source toolkit named Word2vec which was developed by Google in2013. It was used for trans lating the corpus into word vectors [11]. Word vectors are used as training features for text classification by using machine learning algorithms such as Support Vector Machine (SVlv1), Random Forest, and so on.

1) Word2vec Word2vec can simplify the processing of text content into

vector operations in vector space and calculate the similarity in vector space to represent the semantic similarity of the text. Word2vec has two kinds of language models: Continue Bags of Words (CBOW) model and skip-gram model [12]. Both models have input, projection and output layer. The CBOW model predicts the current word based on the context, on the other hand, the skip-gram model predicts the context based on the current word. These two models are shown in Figure 1 and Figure 2:

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:55:45 UTC from IEEE Xplore. Restrictions apply.

Figure 2. Skip-gram model schematic diagram

Wt repr esents the current word.

(2)

Precall + Ppr ecision PFl - score

TP P ---- recall - TP + FN

2 x Ppre cision x Precall

TP +TN Paccura cy :::: -------

TP + TN + FP + FN

TP P . . =--- preCISIOn TP + FP

Un der Curve (AUC) score , Receiver Operati ng Characteristic (ROC) curve , and so on. ROC curve is commonly used in binary class ific ation. AUC score is th e are a of ROC curve. In addition, there are some other important norms . TP rate (True Po sitiv e rate ) r epresent s the per centag e of po sitiv e samples which are predi cted as positive. TN rate (True Negative rate) represents the p ercentage of negativ e samples whi ch are predicted as negative. FP rate (False Positive rate) r epresents the percentage of negative samples which are predic ted as positiv e. FN rate (False Negative rat e) repre sents the per centage of negative samples whi ch are predicted as po sitive by th e mod el [17]. The abscissa and the ordinate of ROC curve is FP and TP rate. The larger AUC is, the better the classification effect. These norms can be represented as follows:

WIt)

OutputProjectionInput

Wlt-2)

Wlt -i}

Figure 1. CBOW model sch ematic diagram

W{tt2)

Input Projec ti on Output

W( t -2}

W( t -l} W It }

W (t t i}

0 W( t+2 )

W{tti}

2) Principal Comp onent Analysi s (PCA)

Pri ncipal Component Analysis (PCA) method is one of the mos t wi dely used in data compression algori thm . After the text wa s tran sferred into word vector s throu gh Word2vec, PCA metho d can reta in the main features while red ucing dimension of data. PCA can remove nois e and unimportant fe atures so as to enhan ce the spee d of data proc essing . The reduc ed dimension is the principal element [13]. The dimensiona lity reduction effect of PC A can be express ed as follow s :

l = min w tr(WT AW) (1)

C. Deep learn ing- based m ethod Deep learnin g approach can classify the sentiment tendency

as well . This p aper adopted Deep Neural Netw ork (DNN) [15] method to perform sentiment analysi s test. DNN establi shes input layer, hidden layer and output lay er which is shown in Figure 3 . The Layers of DNN mode l are ful ly connected. Using the output of the previous layer to calcu late the output of the next laye r is the DNN forward propagation algorithm . The DNN forward propagation algori thm is used to start from the input layer, and the layer-by-layer calcnl ation is performed backwards until the operation rea che s the output layer, then get the final resnlt [16] .

W represents a matrix which contains all map pin g v ectors as the colum n vector s, tr repre sent s the trace of matrix, A repr esen ts data covariance matrix, I repres ents explained variance. The num ber of I rea ches zero means the smallest err or.

3) Ran dom Forest

This paper selec ted a typical classifier, Ra ndom Forest, for training mode l. Random For est clas sification algorithm is an ens emble of Decision Tre e and is a kind of supervise d algorithm. The De cision Tree model is a tree struc ture whi ch is consiste d of nod e and directe d edge . Objects whi ch are need to b e classified are pushe d from the root nod e to th e leaf nod e by performing a binary test in each tree [14].

4) Classifi cati on Performan ce Norm s

To better and mor e obje ctive describ e the analy sis effe cts of the classifiers based on machine learning method, this paper adopted some classification performance norm s such as Area

o utpu t leyc r

Figure 3. DNN schematic diagram

396

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:55:45 UTC from IEEE Xplore. Restrictions apply.

From TABLE I we can obtam the classification indicators of the Random Forest classifier and the ROC curve was shown in Figure 6:

p P,.ecall PFl~score AUCaccuracy

nezative 092 0.84 0.88 nositive 077 0.88 0.82 0.94 ava/total 0.86 0.86 0.86

It can be seen from the Figure 5 that when the number of principle component was started from 100, the explained variance was nearly reached zero which meant the smallest error. Considered the redundant information reduction, we can conclude the best dimension is 100.

To evaluate the classification effects, we chose Random Forest classifier. The corpus was randomly divided into training set and testing set. After training and predicting, we compared the predicted value with the actual value to get the correct rate and some other norms to describe the effect of the Random Forest classifier which was shown in TABLE 1:

III. PERFORMANCE OF SENTIMENT ANALYSIS This paper chose 4000 Chinese hotel reviews which were

came from Ctrip.com and were manually labeled. Positive and negative corpus each had 2000 articles. This paper completed the implementation of three sentiment analysis approaches and compared classification effects. Finally, the machine learning method using random forest algorithm worked best and the correct rate was 85.60%.

A. Lexicon-based method In lexicon-based method, this paper used an open-source

sentiment dictionary which had words and polarity value and degree dictionary. After data preprocessing and reading into the emotional dictionary, degree words and negative words, and comparing with the words in the corpus. The score was increased by I if there was a negative word and was multiplied by the degree word coefficient if there was a degree word and plus the emotional word polarity if there was an emotional word. Finally, by comparing the scores, greater than 0 meant positive evaluation, less than 0 meant negative evaluation, and the absolute value of the score represented the degree. The result was shown in Figure 4:

TABLE L THE CLASSIFICATION PERFORMANCE EFFECT OF RANDOM FOREST CLASSIFIER

In lt l : run co d e. py Bui ldmg pa ef rx da c t fr oln t h e def au lt dictl onary . •.

l oa di n g model f r om c ache C: \Use rs \S 12\ AppData \ Loca l \ TE' lnp\ J i eb a . c ache

l oa ding model cos t 0 .83';" s e cond s.

Prefix di et has be en bt uI t succesf ully.

corr ec t r at e : 0.55810 16393 -1 -12623

Figure 4. The result oflexicon-based method

l..o r0.8 ~ 'a ~ 06

j ~ 0.4

The final correct rate oflexicon-based method was 55.85%. 0.2

Figure 6. The ROC curve of Random Forest classifier

It can be seen from Figure 6 that the AUC value is 0.94.

C. Deep learning-based method In deep learning-based method, this paper used Keras

library which can be based on tensorflow in python environment to build DNN model frame. We adopted two hidden layers and selected the number of iterations named epoch was 500. After training, the test correct rate reached 85.09% and the training process curve was shown in Figure 7:

i.o0.8 - area - 0.94

0.'0.4 True Positive value

0.2 B. Machine learning-based method

In machine learning-based method, the corpus needed to be first preprocessed such as word segmentation, remove stop words. We used the open-source and trained Wikipedia word2vec model to vectorize the corpus. Then the PCA algorithm was used to choose the best dimension through explained variance. The effect diagram was shown as Figure 5:

a a 100 200 300

prin cipal com po ne nt 400

Figure 5. explained variance curve with increasing principal component

397

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:55:45 UTC from IEEE Xplore. Restrictions apply.

The Comparison of the three methods and their classification correct rate was shown in TABLE II:

method lexicon Machine

Deep learning learnins

correct rate 55.85% 85.60% 85.09% From TABLE II we can conclude that the Random Forest

algorithm based on machine learning method and deep learning method are better that lexicon-based method.

It can be seen from Figure 7 that with the increase of the number of iterations, the classification correct rate can reach up to 85.09%.

words, II 2017 International Conference on Electri cal, Computer and Conununication Engineering (ECCE), Cox's Bazar, 2017, pp. 186-190.

[4] R. Jose and V. S. Chooralil, "Prediction of election result by enhanced sentiment analysis on twitter data using classifier ensemble Approach," 2016 International Conference on Data Mining and Advanced Computing (SAPIENCE), EmakuIam, 2016, pp. 64-67.

[5] 1. Li and L. Qiu, "A Sentiment Analysis Method of Short Texts in Micrcblog," 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUe), Guangzhou, 2017, pp. 776-779.

[6] W. Zongyue and Q. Sujuan, "A sentiment analysis method of Chinese specialized field short commentary," 2017 3rd IEEE International Conference on Computer and Conununications (ICCe), Chengdu, 2017, pp.2528-2531.

[7] M. Maia, A. Freitas and S. Handschuh, "FinSSLx: A Sentiment Analysis Model for the Financial Domain Using Text Simplification," 2018 IEEE 12th International Conference on Semantic Computing (ICSC), Laguna Hills, CA, 2018, pp. 318-319.

[8] W. Cheng, Y. Song, Y . Zhu and P. Jian, "Dimensional Sentiment Analysis for Chinese words Based on synonym lexicon and Word Embedding," 2016 International Conference on Asian Language Processing (IALP), Tainan, 2016, pp. 312-316.

[9] Z. Fan, L. Su, X. Liu and S. Wang, "Multi -label Chinese question classification based on word2vec," 2017 4th International Conference on Systems and Informatics (ICSAI), Hangzhou, 20 17,pp. 546-550.

[10] N. Chirawichitchai, "Emotion classification of Thai text based using term weighting and machine learning techniques," 2014 11th International Joint Conference on Computer Science and Software Engineering (JCSSE), Chon Buri, 2014 , pp- 91-96.

[11] A. H. Ombabi, o. Lazzez, W. Ouarda and A. M. Alimi, "Deep learning framework based on Word2Vec and CNN for users' interests classification," 2017 Sudan Conference on Computer Science and Information Technology (SCCSIT), Elnihood, 2017, pp- 1-7.

[12] Zhi-Tong Yang and Jun Zheng, "Research on Chinese text classification based on Word2vec," 2016 2nd IEEE International Conference on Computer and Conununications (ICCe), Chengdu, 2016, pp. 1166-1170.

[13] Lu Ye, Rui-Feng Xu and Jun Xu, "Emotion prediction of news articles from reader's perspective based on multi-label classification," 2012 International Conference on Machine Learning and Cybernetics, Xian, 2012, pp. 2019-2024.

[14] S. Sedai, P. K. Roy and R. Gamavi, "Right ventricle landmark detection using multiscale HOG and random forest classifier," 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), New York, NY, 2015, pp. 814-818.

[15] Y. Pei, X. Hongyan and D. Yuan, "Classification of marine noise signals based on DNN (Deep Neural Networks) model," 2017 13th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), Yangzhou, 2017, pp. 465-470.

[16] Deep neural network (DNN) model and forward propagation algoritlun. Jianping Liu Pinard. https://www.cnb1ogs.comlpinardlp/6418668.html

[17] L Hidayah, A Erna P. and M. A Kristy, "Application of J48 and bagging for classification of vertebral co1unm pathologies," Proceedings of the 6th International Conference on Information Technology and Multimedia, Putrajaya, 2014, pp. 314-317.

>0,'"'"'epoch'"

COMPARISON OF DIFFERENT CLASSIFIERS

'"

Figure 7. The training process curve

I - val ace

~ r '.,. ,

~

-:[ , -j- ,...J

0.55

TABLE II.

IV. CONCLUSION

The result of this paper was that the deep learning method was slightly worse than the machine learning method, mainly because the amount of data was insufficient. In the future, we will collect more data for further research on deep learning method. In the future, the main work is to increase the amount of data for model training and prediction, and the deep learning method is estimated to achieve better results.

R EFERENCES

[1] K. S. Sabra, R. N. Zantout, M. A. E. Abed and L. Hamandi, "Sentiment analysis: Arabic sentiment lexicons," 2017 Sensors Networks Smart and Emerging Teclmologies (SEN SET), Beirut, 2017 , pp. 1-4.

[2] Y. Woldemariam, "Sentiment analysis in a cross-media analysis framework," 2016 IEEE International Conference on Big Data Analysis (ICBDA), Hangzhou, 2016, pp. 1-5.

[3] M. AI-Amin, M. S. Islam and S. Das Uzzal, "Sentiment analysis of Bengali conunents with Word2Vec and sentiment information of

0.60

0.65

0.80

085

0.70

0 .15

398

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:55:45 UTC from IEEE Xplore. Restrictions apply.