help

2018 Conference on Informati on Communi cati ons Technol ogy and Soci ety (ICTAS)

A Framework for Sentiment Analysis with Opinion Mining of Hotel Reviews

Kudakwashe Zvarevashe ICT and Society Research Group, Durban University of Technology,

P.O. Box 1334, Durban 4000, South Africa [email protected]

Oludayo O Olugbara ICT and Society Research Group, Durban University of Technology,

P.O. Box 1334, Durban 4000, South Africa [email protected]

Abstract — The rapid increase in mountains of unstructured textual data accompanied by proliferation of tools to analyse them has opened up great o ppo rtunities and challenges for text mining research. The automatic labelling of text data is hard because people often express opinions in complex ways that are sometimes difficult to comprehend. The labelling process involves huge amount of efforts and mislabelled datasets usually lead to incorrect decisions. In this paper, we design a framework for sentiment analysis with opinion mining for the case of hotel customer feedback. Most a vailable datasets of hotel reviews are not labelled which presents a lot of works for researchers as far as text data pre-processing task is concerned. Moreover, sentiment datasets are often highly domain sensitive and hard to create because sentiments are feelings such as emotions, attitudes and opinions that are commonly rife with idioms, onomatopoeias, homophones, phonemes, alliterations and acronyms. The proposed framework is termed sentiment polarity that automatically prepares a sentiment dataset for training and testing to extract unbiase d opinions of hotel services from reviews. A comparati ve analysis was established with Naïve Bayes multinomial, sequential minimal optimization, compliment Naïve Bayes and Composite hypercubes on iterated random projections to discover a suitable machine learning algorithm for the classification component of the framework.

Keyword s: opinion mining, sentiment analysis, machine learning algorithm, natural language processing, sentiment polarity, dataset labelling

I. INTRODUCT ION In recent years, the world has experienced a tremendous

rise in the volu me of te xtual data especially for the unstructured data generated from people who e xpress opinions through various web and social media platforms for diffe rent reasons. Mountains of these textual data, in itia lly could be equated to garbage which would need to be disposed fro m t ime to time. However, with the advancement in storage capacity accompanied by the increasing sophistication in data mining tools, opportunities and challenges have been created for analysing and deriv ing useful insights from these mountains of data.

In this paper, we have chosen textual data in the form of hotel reviews for sentiment analysis with opinion min ing fro m customer perspectives. Sentiment analysis uses the techniques of natural language processing and computational linguistics to automate the classification of sentiments generated from reviews. Hotels provide satisfaction, security,

ISBN 978-1-5386-1001-5/ 18/ $31.00 ©2018 IEEE

comfo rt, lu xu ry and lodging services for travellers and people on vacation. Mining hotel revie ws is desirable to gain deeper knowledge of customer e xpectations and support effective manage ment of customer re lationships. It would enable the hotel managers to have a good understanding of customer needs, discover areas for further improve ment and improve service quality. The hotel reviews are provided exclusively by customers who have made reservations at a particular hotel. Customers post feedback about hotels which include hygiene, quality of food, location, customer service quality and hospitality e xh ibited by hotel staff. Moreover, sentiment analysis of hotel revie ws is c rucia l to understand hidden patterns generated by data that would help to effect ively improve performance [1].

II. RELATED LITERATURE

Sentiment analysis [2] and opinion mining [3] a re terms that refer to the field of study that analyses opinions, evaluations, appraisals, attitudes and emotions of people towards entities such as products, services, organizations, individuals, issues, events, topics and their attributes [4]. These terms we re used interchangeably to define opinions that entail positive or negative sentiments [4, 5]. Sentiment determines the polarity of opinions expressed in a given review. Ca mbria et al. [6] disputed the interchange of these concepts by classifying opinion mining as polarity detec tion and sentiment analysis as focusing on emotion recognition. The opinion min ing system only needs to understand polarity that can be positive, negative or neutral sentiments depending on the nature of sentences expressed in a revie w [7]. The process of detecting polarity is strongly linked to analysing sentiments on a particular subject.

Most researches on sentiment analysis are focused on descriptive data. Manke and Shivale [8] e xp lored the significance of social networks as preferred environ ments for opinion mining and sentiment analysis. They introduced the original method of opinion classification and tested their algorith m on rea l social network datasets. They concluded fro m their findings that social networks e xhib it properties that make them suitable for opin ion min ing activities. Co mprehensive surveys have been presented on various methods used in opinion mining [9-11] with limited focus on aspect oriented analysis.

The ma jority of current methods of sentiment analysis attempt to detect the polarity of a review regardless of the

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:45 UTC from IEEE Xplore. Restrictions apply.

entities such as hotels and facilities with their respective aspects such as food and internet access for instance. By contrast, the task of this study is concerned with aspect based sentiment analysis with the goal of identifying the aspects of given target entities and sentiment e xpressed towards each aspect. The aspect based sentiment analysis summa rizes what people like and dislike fro m reviews of products or services. It has always been a difficu lt task [7] because several subtasks such as feature extract ion, feature grouping, polarity classification and evaluation measures have to be performed to get an unbiased opinion, usually under the assumption of grammar free errors which is not always realistic.

Hotel revie w is an important theme of natural language processing (NLP), which is a discipline that deals with processing of textual data [12]. It is at the intersection of artific ial intelligence and linguistics [6] which ma kes NLP techniques amenable to sentiment analysis. The analysis of sentiment can be performed using two basic approaches of le xicon-based and machine learn ing [9, 10]. The mach ine learning approach which this study is based upon utilizes t wo ma in learn ing techniques of supervised and unsupervised learning. Supervised learning a lgorith ms such as the support vector mach ine, Naïve Bayes, K-nearest neighbour and convolutional neural network deep learn ing have been applied for sentiment analysis. Supervised learning algorith ms require the train ing of machine using labelled dataset. However, unsupervised learning algorith ms such as the K-means and Fuzzy C-means clustering do not require training datasets because they learn by observation.

The application of supervised learning a lgorith ms as applied in this study to volumes of labelled train ing data on hotel reviews can provide insightful in formation that would help hotels imp rove performance and overall ratings a mongst competitors. Different evaluation measures such as true positive rate, fa lse positive rate, prec ision, reca ll, and F- measure rate, receiver operating characteristic (ROC) area and precision recall curve (PRC) area are the benchmark metrics used to determine the accuracy of different supervised learning a lgorith ms. However, to get accurate results of these measures, there is the need to design a fra me work that fosters the automatic creat ion of properly labelled datasets containing actual sentiments expressed by customers.

The success of sentiment analysis and opinion mining hinges largely on the use of tools for e xecuting d ifferent NLP tasks. Tools that can be used for the NLP tasks include Red Opal wh ich is used for enabling users to find products based on aspects. Tools that are used to help companies e xtract and analyse opinions of customers on products from blogs include Sentic Net, Luminoso, Factiva, Attensity and Converseon. The NLTK, OpenNLP and Stanford Core NLP are widely used NLP toolkits to support the imple mentation of basic NLP tasks such as POS tagging, na med entity recognition and parsing [11]. The W EKA system is a wide ly used tool that contains a collection of techniques for data analysis and predictive modelling. It supports several standard data mining tasks that include data pre-processing,

clustering, classification, regression, visualization, association rule min ing and feature selection [13]. Data can be imported fro m an e xterna l source in formats such as comma separated value (CSV) file . The data in its raw format needs to be cleaned as it may not be compatible with the processing required. Pre-processing is done to transform the raw data into a format that can be manipulated by appropriate tools. Furthermore , the la rger the dataset, the more accurate is the performance of the lean ing algorith ms that is inherently measured in terms of the standard evaluation metrics.

III. METHODOLOGY

A. The Intuition Model

The conceptual view of the intuit ion model shown in Figure 1, begins with the feedback collection. Customers respond to questionnaires concerning their fee lings about services received from the selected hotels. This can be done in a nu mber of ways, for e xa mple opening a web porta l through which customers can drop comments. The next step will be to label the comments based on intuition. This will be done by human agents who simply read the co mments and assign labels based on perceptions. Once data a re transformed to a desire format, the ne xt step will be to convert the labelled text to feature vectors through the use of filters. This will ma ke it easier to imp le ment a classification algorith m for training and testing of data. The next step involves the selection of an appropriate classificat ion algorith m whilst the last step is the training and testing of the selected algorith m on dataset and capturing of results.

B. The Sentiment Polarity Based Model

The research reported in this paper was done using the sentimental pola rity based model (SPBM ) as illustrated in Figure 2. Just like the intuition based model (IBM), the SPBM mode l begins with the elic itation of opinions which is the step skipped because we used the raw OpinRank dataset [14, 15]. Customers respond to questionnaires concerning services received fro m the selected hotels through an appropriate user interface. The ne xt step will be to label the comments based on sentiment polarity score using a sentiment polarity a lgorith m. The score obtained will determine whether a co mment is positive, negative or neutral. Once the data are transformed, the ne xt step is to convert the labelled te xt to feature vectors through the use of filters. The next step involves the selection of a suitable classification algorith m. The last step is the training and testing of the selected classification algorithm and capturing of results. The distinguishing property of SPBM is labe lling is that automatic, it does not involve human intervention and it is quite consistent in labelling sentiments. However, the IBM relies heavily on human intervention to label sentiments which sometimes may not be consistent and the labelling process is intrinsically laborious and time demanding.

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:45 UTC from IEEE Xplore. Restrictions apply.

IV. EXPERIM ENTA L RESULTS Data fro m the Opin Rank opin ion based ranking dataset

was acquired to e xperimentally test the performance of the investigated learning a lgorith ms to discover a suitable algorith m for the classificat ion component of the SPBM fra me work. We selected the OpinRan k dataset because it contains unlabelled reviews that gave the fle xib ility for a custom e xperimentation. The Opin Rank dataset contains approximately 259000 unlabelled reviews on cars and hotels fro m 80 to 100 hotels in 10 different cities across the world. These cities include Dubai, Beijing, London, New York City, New De lhi, San Francisco, Shanghai, Montreal, Las Vegas and Chicago. We selected hotel reviews fro m London, Be ijing and Montreal as a matter of choice to c reate a sub- dataset for our experimentation. We labelled the hotel features in the dataset using a sentiment polarity software written in Python with the TextBlob which is a libra ry for processing textual data. The scores obtained from the sentiment polarity we re then used to automatically labe l the data. After performing all the necessary processing s teps, including labelling and filtering, the dataset was split into two subsets to create testing and training datasets. We used four classification algorithms wh ich are Naïve Bayes mu ltino mia l (NBM), Sequential min ima l optimization (SMO), Co mpliment Naïve Bayes (CNB) and Co mposite hypercubes on iterated random projections (CHIRP) to train and test the dataset. Figure 3 shows the co mparative results obtained after e xperimentation. The Naïve Bayes mult inomia l a lgorith m had the highest precision which reached 80.9% . It was closely followed by the compliment Na ïve Bayes algorithm which had 80.5%. CHIRP was the lowest performing algorithm and it scored 75.6% in precision.

V. CONCLUSION

The s entiment analysis with opinion mining fra mework reported in this paper can be incorporated into a hotel technology system that can help improve customer relationship management. What good is a system that predicts the polarity of sentiments if it wo rks with the wrongly labelled data? Fro m the sentiment polarity e xerc ise that we did, we found out that some co mments may be wrongly viewed as neutral while they will be either positive or negative. The following e xa mp le was viewed as a neutral comment. “That hotel is surely a HELLT EL!” This comment is truly negative and sarcastic, but because the word HELLT EL does not exist in the English vocabulary it was classified under the neutral class. However, most comment s were labelled with a much better accuracy. We believe that a lot of research can be done in this area especially in fine tuning the feature extract ion algorith m of the fra me work so that classification error is minimised.

The system is e xpected to determine sentiments the way human beings do and labelled datasets are norma lly used for the system to lea rn automat ically. The proposed fra mework tries to ma ke sure that sentences are correctly labelled such that false information is not fed into the system. In a nutshell, the proposed framework in this paper helps in automatic

labelling of sentiment datasets. The e xperimental result of this study has indicated that Na ïve Bayes multino mial algorith m gave good performance when compared to other classification algorithms in terms of the evaluation metrics applied. In future work, we would like to improve on automatic labe lling, feature e xt raction and perform classification of customer responses based on emotions using deep learning algorithms.

ACKNOWLEDGMENT We would like to thank Prudence Kadebu and Innocent

Mapanga from Harare Institute of Technology, Belvedere, Harare for their heartfelt assistance.

REFERENCES [1] V. Dhanalakshmi, B. Dhivya and A.M. Saravanan, “ Opinion mining

from st udent feedback data using supervised learning algorithms”. IEEE 3rd MEC International Conference on Big Data and Smart City. , pp. 1-5, 2016.

[2] J. Yi, T . Nasukawa, R. Bunescu and W. Niblack, “ Sent iment analyzer: Extract ing sent iment s about a given topic using nat ural language processing techniques”. In Data Mining, 2003. ICDM 2003. Third IEEE International Conference, pp. 427-434, 2003.

[3] K. Dave, S. Lawrence D.M. and P ennock, “Mining t he peanut gallery: opinion extract ion and semant ic classificat ion of product reviews”. In ACM Proceedings of the 12th international Conference on World Wide Web, pp. 519-528, 2003.

[4] B. Liu, “ Sent iment analysis and opinion mining”. Synthesis Lectures on Hum an Language Technologies, vol. 5, no. 1, pp. 1-167, 2012.

[5] A. Buche, D. Chandak and A. Zadgaonkar, “ Opinion mining and analysis: a survey”. International Journal on Natural Language Com puting (IJNLC), vol. 2, no. 3, pp. 39-48, 2013.

[6] E. Cambria, B. Schuller, Y. Xia and C. Havasi, “New avenues in opinion mining and sent iment analysis”. IEEE Intelligent Systems, vol. 28, no. 2, pp. 15-21, 2013.

[7] M.S. Akht ar, D.k Gupt a, A. Ekbal and P . Bhattacharyya. "Feat ure select ion and ensemble const ruct ion: a t wo-st ep method for aspect based sent iment analysis." Knowledge-Based Systems, vol. 125, pp. 116-135, 2017.

[8] S.N. Manke and N. Shivale, “ A review on: opinion mining and sent iment analysis based on nat ural language processing”. International Journal of Computer Applications, vol. 109, no. 4, pp. 29-32, 2015.

[9] W. Medhat , A. Hassan and H. Korashy, “ Sent iment analysis algorithms and applicat ions: a survey”. Ain Shams Engineering Journal, vol. 5, no. 4, pp. 1093-1113, 2014.

[10] J.A. Balazs and J.D. Velásquez, “Opinion mining and informat ion fusion: a survey”. Information Fusion, vol. 27, pp. 95-110, 2016.

[11] S. Sun, C. Luo and J. Chen, “ A review of nat ural language processing t echniques for opinion mining syst ems”. Information Fusion, vol. 36, pp.10-25, 2017.

[12] P .M. Nadkarni, L. Ohno-Machado and W.W. Chapman, “Nat ural language processing: an int roduct ion”. Journal of the Am erican Medical Informatics Association, vol. 18, no. 5, pp. 544-551, 2011.

[13] M.T. Khan, M. Durrani, A. Ali, I. Inayat , S. Khalid and K.H. Khan, “ Sent iment analysis and the complex nat ural language”. Complex Adaptive Systems Modeling, vol. 4, no. 1, pp. 1-19, 2016.

[14] M. Hall, E. Frank, G. Holmes, B. P fahringer, P . Reut emann and I.H. Witt en, “The WEKA data mining soft ware: an updat e”. ACM SIGKDD explorations newsletter, vol. 11, no. 1, pp. 10-18, 2009.

[15] K. Ganesan and C. Zhai, “ Opinion -based ent ity ranking”. Information retrieval, vol. 15, no. 2, pp. 116-150, 2012.

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:45 UTC from IEEE Xplore. Restrictions apply.

Figure 2. Sentiment polarity analysis framework

Figure 1. Intuitive sentiment analysis framework

Figure 3. Weighted average score against performance metrics

NBM SMO CNB CHI RP

0.90 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00

Performance metri c

W ei

gh te

d a

ve ra

ge s

co re

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:45 UTC from IEEE Xplore. Restrictions apply.