help

The 6th International Conference on Cyber and IT Service Management (CITSM 2018) Inna Parapat Hotel – Medan, August 7-9, 2018

Sentiment Analysis of Online Auction Service Quality on Twitter Data: A case of E-Bay

Calandra Alencia

Haryani Fakultas Ilmu Komputer

Universitas Indonesia Jakarta, Indonesia

[email protected]

Achmad Nizar Hidayanto

Fakultas Ilmu Komputer Universitas Indonesia

Jakarta, Indonesia [email protected]

Nur Fitriah Ayuning Budi

Fakultas Ilmu Komputer Universitas Indonesia

Jakarta, Indonesia [email protected]

Herkules Fakultas Ilmu Komputer

STMIK Palangkaraya Jakarta, Indonesia

[email protected]

Abstract- Customer Satisfaction is the most important factor to determine long term success of a company, and that includes Online Auction company. Customer Satisfaction is important for an Online Auction company to enhance its customer loyalty in the middle of today’s competition. This research is done to analyze the significant Service Quality of Online Auction, using sentiment analysis in Twitter, by targeting the customers that used online auction E-Bay; one of the largest online auction platform in the world. The data is retrieved from tweets of E-Bay’s customer to @Ebay and @askEbay twitter account, before being processed using Lexicon Classification method to generate sentiment analysis for each significant service quality factors. The Results shows that Information Reliability, Interface Design and Security, and Reliability are the factors that gained the most feedback and sentiment from customers.

Keywords: online auction, E-Bay, customer satisfaction, service quality, lexicon classification, sentiment

I. INTRODUCTION

Electronic auction is a platform from Electronic Commerce (E-Commerce) that has a purpose to become a media for auction transaction between auctioneer and bidder (Customer to Customer) in non-physical media, such as internet [1]. Electronic auction aids to facilitate auctioneer (seller) and bidder (buyer) electronically, in which the auctioneer and bidder can be done by anyone who can deal anything if it’s considered legal.

In today’s emergence of online industry, customers have many E-Commerce platforms to choose. For instance, E-Bay, the pioneer of online auction, has its own competitor such as Priceline. This situation requires their stakeholder to do various action to maintain its customer, specifically by enhancing its customer service quality. [3].

The performance of service quality has direct impact to customer satisfaction. The service quality excellence is measured by company’s performance to fulfill customer expectation [2] [3]. Fulfilled customer satisfaction would impact trust and loyalty of users to the electronic platform [4]. Research shown that there is a positive relationship between non-financial measure such as customer satisfaction with financial performance that is sales and profit growth in a company. Moreover, customer satisfaction could reduce

customer intention to switch to another platform and product [3].

It is important for the stakeholder of E-Commerce platform to evaluate their customer satisfaction to its service. The most commonly used approach is by using questioner, which implicate different kind of dimension such as information reliability, interface design, website timeliness, security and privacy, and customer service [5] [6] [7] [8] [9] [10] [11]. The emergence of social media usage resulted in the tendency of customers to share their comment about specific product or service in their social media which could be accessed by anyone; this information could become valuable tool to be evaluated.

The purpose of this research is to evaluate customer satisfaction to online auction platform using sentiment analysis to E-Bay user. E-Bay is chosen in this research, based on information that is reported in Statista in 2017, that E-Bay is one of the largest E-Commerce company in America. The company is founded in 1995, which has gross transaction around 9.6 billion USD or around 304 USD per second in 2017. Until the 4th Quarter of 2017, E-Bay has 170 millions of active auctioneer (seller) [12].

II. LITERATURE REVIEW

A. Online Auction Online auction is a form of E-Commerce that provide service

to do auction transaction through Internet as a medium. Auction is an activity of buying and selling where the seller sets a minimum price in a set period and the buyer could bid for a price higher than buyer set price or other bidder; which then the goods or service will be delivered to the highest bidder when the set-time runs out [13]. Examples of popular online auction platform are e-Bay, Priceline, etc.

One of the largest online platform is E-Bay. Their Business model is to ensure online transaction between auctioneer (seller) and bidder (buyer) succeed [14]. The service they provide to seller [14]: providing different kind of products and services to be sold (new, refurbished, secondhand), selling method choice (fix price or auction), etc. On the buyer side, the service are [14]: purchasing different kind of goods (new, refurbished, secondhand), shipping method (pickup, delivery courier), and etc.

The 6th International Conference on Cyber and IT Service Management (CITSM 2018) Inna Parapat Hotel – Medan, August 7-9, 2018

B. Service Quality for Online Auction E-Bay

Service quality is a quality of interaction, information source, and internet-based services that are integrated with terms and condition with a purpose to strengthen the relationship of customer service to enhance their trust towards the company [15].

Literature review study towards factors that could determine positive and negative sentiment of online auction customer that has influence to its services are: [5] [6] [7] [8] [9] [10] [11] 1. Information Reliability encapsulate information about

products offered in the website. Customer needs information that is accurate, trustworthy, and quick about specific product offered with little effort to lessen doubt and enhance trust [16] [17].

2. Interface Design includes how satisfied users towards the interface of the website which consist of images, text, and layout for each website pages. Well-formatted website could effectively and efficiently provide information needed by users [18] [19]. Ease of use which is a quality of how well the website perform its operational characteristic, is a part of Interface Design Service Quality [20] [21]. Ease to understand product helps users to use it fast, precise, ease, and without mistake [22], It is also important to have ease of navigation to different website pages during online auction process [11] [23].

3. Website Timeliness indicate real-time information about bidder status of before, during, and after bidding [24] [17]. Website inability to provide real-time and fast update would upset its customer’s user experience as it is part of the essence of online auction.

4. Security and Privacy is a personal data privacy for all users registered in auction and also the security during financial transaction [24] [17]. The Security envisioned is a form of structured protocols and procedures to ensure the auction products delivered from auctioneer (seller) to bidder (buyer) [25] [26]. Well-managed Security privacy could impact customer choice and trust towards online platform.

5. Customer Service refers to how well customer service assist, responsible, and aid customer complaint [6] [8]. Good communication service might provide security and comfort for the users and contribute to customers’ trust and satisfaction.

All the things mentioned above are the elements of service quality related to customer satisfaction towards Online Auction Platform. C. Sentiment Analysis

Sentiment analysis is an analytic subject to discover an opinion, assessment, attitude, and emotion of public towards specific object such as product, service, organization, individual, issue, event, topic, etc. [27]. The result of Public Opinion is in a form of positive or negative sentiment. From then, correction and improvement are done based on these objects to maintain or enhance customer satisfaction. A study by Bo Pang and Lilian Lee who done research about Opinion Mining and Sentiment Analysis using various classification technique such as Lexicon Classification,

Bootstrapping, and Opinion Finder [28] is one of example of Sentiment Analysis use.

III. RESEARCH METHODOLOGY

This research is done using R Language application, the process methodology is consisting of data collection, data cleaning, and lexicon classification. R Language is the application to perform sentiment analysis, as this application could gather, process, classify, analyze, and do conditional lopping pf data based on different kind of various desired input [29]. A. Data Collection

Raw data is collected from English language tweets from to all users that mentioned @Ebay and @askEbay. Using R Studio, data crawling is performed from social media: Twitter in a form of comments that being copied and retrieved to the application based on specific word related to service quality contained; before then being cleanse and classify in the next steps. B. Data Cleaning

Fig. 1. Data Cleansing Steps

Data Cleansing Process has a purpose to cleanse raw data

gathered during data collection process. The Cleansing Process is done by removing syntaxes and unnecessary characters to ensure mistake will not happen and speed up classification and analysis process. The platform used in data cleansing process is R Studio, the steps are as follows [30]: 1. Type Checking and Normalizing Normalizing process is done by identifying each character

and its data type, and then, removing exceed spaces, special (non-alphabetical) character, and additional unmeaningful characters; to be technically processed.

2. Fixing and Connecting Normalized data is then being fixed if there is a typo and sorted according to requirement to eliminate inconsistency between words.

The 6th International Conference on Cyber and IT Service Management (CITSM 2018) Inna Parapat Hotel – Medan, August 7-9, 2018

3. Estimation and analyze

Each consistent word will then be being combined and analyzed in the next sentiment analysis step.

C. Lexicon Classification

Classification is completed by grouping every sentiment analysis towards each service quality factors discussed in literature review. Each word gathered during data collection, related to intended service quality is being grouped and calculated its frequency.

TABLE 1. Service Quality Dictionary

Quality Service Dictionary

Information Reliability

integrity, plenitude, all, completeness, whole, entire, essential, up to date, advance, modern, current, fitting, happening, new, newest, suitable, accurate, certain, sure, precise, structured, efficient, fixed, framed, meaningful, important.

Interface Design

convenient, handy, useful, access, easy to understand, feasible, simple, layout, beautiful, neat. acceleration, agility, momentum, pace, easy to operate, adoption, help, handling, practice, easy to use, easy

Timeliness

response, responsive, quickness, velocity, activity, fast, quick in browsing, quick, real time, reliable, connection.

Security and Privacy

secure, safe, encrypted, controlled, authority, personalized, restriction, regulation, personal, insurance, sound, agreement, lock, locked, warranty, custom, reliable, safe, solid, sure, stable, steady, relevant, useful, valid,

Customer Service

24-hour service, continuous, endless, ongoing, constant, nonstop, day and night, all day, at all times, perpetual, never ending, everlasting, response, after sales service, client service, help line, customer service, cs, product service, troubleshooting, response, support, client service, help line, customer service, support

Steps to classify category specific lexicon analysis [31] [32]

[33]: 1. Every Service Quality is collected from analyzing sentences

that contained words that has already been categorized and contained relation with corresponding Service Quality.

2. Every word is grouped according to positive and negative category based on dictionary “Sentiment-Lexicon” [34].

3. Every word that has been classified and grouped, is being counted its frequency and then analyzed according to the most relatable factors to customer satisfaction

4. The Result of the calculation is used to determine what are the factors that has impact to customer satisfaction towards e-commerce.

IV. RESULT A. Data Crawling

TABLE 2. Data Crawling Quality Service

Quality Service Number of Tweets Information Reliability 93.895 Interface Design 145.885 Timeliness 8.017 Security and Privacy 24.559 Customer Service 10.568

The research use quality service dictionary to perform data crawling in Twitter social media using R Language. The Data crawling process resulted 281.926 tweets from users who mentioned @Ebay and @askEbay. The Most tweets came from Interface Design topics with 145.885 tweets or around 51% of total data gathered. The second most talked tweets is about Information Reliability with 93.895 tweets or around 34% of total data gathered. Then, the Third, Forth, and Fifth factors discussed is about Security and Privacy, Customer Service, and Timeliness, with quite significant difference with 24.550 tweets, 10.568 tweets, and 7.019 tweets respectively; where the percentage contributed from total data are 9%, 4% and 2% respectively.

Fig. 2. Data Crawling Graph

B. Customer Satisfaction Sentiment Analysis

TABLE 3.

Sentiment Analysis Table Quality Service Positive Negative Sentiment

Information Reliability

41.305 14.806 26.499

Interface Design 17.690 2.961 14.729 Timeliness 7.231 786 6.445 Security and Privacy

18.702 3.907 14.795

Customer Service 6.670 1.593 5.077

Based on 93.895 data crawled regarding customer satisfaction about Information Reliability, only 56.111 or around 59.76%, there is 41.305 positive sentiment and 14.806 negative sentiment; proportion of 73.61% positive sentiment to 26.39% negative sentiment. This shown that Information Reliability is the most complained service quality factor, although the number of negative sentiment is still below its positive sentiment. This research indicates that online auction user place quite high concern regarding accuracy and presentation of information provided by E-Bay, because accurate and prompt information helps the user to quickly decided if the product auctioned or sold is worth of purchase.

Although sentiment towards Interface Design is the most data crawled, it only generates 14.729 sentiment, which is 10% of total data gathered, with the comparison of 17.690 or 85.66%

The 6th International Conference on Cyber and IT Service Management (CITSM 2018) Inna Parapat Hotel – Medan, August 7-9, 2018

positive sentiment to 2.961 or 14.34% negative sentiment. Overall, the customers satisfied with the convenience and practicality of Interface Design provided by E-Bay. The satisfaction towards design interface is expected to encourage customer intention and reduce its doubt towards online auction service, therefore it convince customers loyalty towards this online auction platform [15].

Sentiment towards timeliness resulted in 100% sentiment feedback from overall data crawling regarding this service quality factor. Positive sentiment result is 7.231 or 90.2% from overall data, while the rest 786 or 9.8% of data is negative sentiment. Overall, the service quality factor of Timeliness is positive sentiment. It can be seen from overall domination of positive sentiment with more than 90% of overall data compared to negative sentiment.

Sentiment towards Security and Privacy resulted in 60% feedback from overall data crawling regarding this service quality factor. Positive sentiment result is 18.702 or 82.7% from overall data, while the rest 3.907 or 17.3% of data is negative sentiment towards Security and Privacy. The comparison shows that customer is quite satisfied with this Security and Privacy service quality in E-Bay as it gained more than 80% positive sentiment. The users believe that eBay is a safe platform and can securely protect their data and transaction which is a user concern towards online transaction [24]

Sentiment towards Customer Service, resulted in 48% of overall Data Crawling. Its Positive Sentiment dominated this factor with 6.670 tweets or 80.7% of overall sentiment compared to 1.593 tweets or 19.3% of overall sentiment analysis towards Customer Service. Therefore, E-Bay has provided adequate service towards its customer, the service quality that is proved to be essential factor to customer loyalty [4] [35].

Fig. 3. Customer Satisfaction Sentiment Analysis Graph

V. CONCLUSION AND RECOMMENDATION

Based on sentiment analysis towards world’s largest online

auction company: E-Bay; it can be concluded that there are a few service quality factors that influence customer’s satisfaction: Information Reliability, Interface Design, and Security and Privacy. These three factors are the most discussed

factor which gain the most positive sentiment from its user, that is the difference between the whole positive sentiment towards negative sentiment related to each service quality factors, in this case, these three factors shown considerable amount of positive sentiment. Which proved that these three factors are the element of service quality that influenced and determined customer satisfaction towards e-auction company.

Online auction company should enhance its service quality, especially regarding Information Reliability. Which although gain the most positive sentiment compared to other factors, it also received the most negative sentiment with 26.30%, and if it’s is compared with the other less service quality: Interface Design, and Timeliness, they only get 14.34% and 9.8% of negative sentiment. The improvement of Information Reliability could be done by enhancing its data collection levels, providing standardized format of information to be filled, and by updating the entire product information periodically.

For future research, sentiment analysis could be done in other media such as E-Bay official webpage, so that the data gathered, and the result would be more comprehensive. Furthermore, clustering process is needed towards the tweets gathered to identify more suitable dimension to represent the service quality of online auction that influence customer satisfaction.

ACKNOWLEDGEMENT This study was supported by PITTA Research Grant No 1862/UN2.R3.1/HKP.05.00/2018. PITTA Research grant was provided by Direktorat Riset dan Pengabdian Masyarakat (DRPM), Universitas Indonesia

REFERENCES

[1] C. C. Aggrawal and P. S. Yu, Privacy-Preserving Data Mining:Model and Algorithms, 2008.

[2] A. G. Awan and M. Azhar, "Consumer Behavior Towards Islamic Banking in Pakistan," European Journal of Accounting Auditing and Finance Research, 2014.

[3] P. Kotler and K. L. Keller, Marketing Management, Pearson Prentice Hall, 2009.

[4] C. Zhang and F. Pan, "The Impact of Customer Satisfaction on Profitability: A study of State owned enterpeise in China," Service Science, vol. 1, no. 1, 2009.

[5] olfinbarger and M. Gilly, "E-TailQ: Dimensionalizing, ring and Predicting Etail Quality," Journal of Retailing, , pp. 183-198, 2003.

[6] J. Field, G. Heim and K. Sinha, "Managing quality in the e-service system:development and application of a process model," Production and Operation management, vol. 13, no. 4, 2004.

[7] A. Parasuraman, V. Zeithaml and A. Malhorta, "E-S- QUAL: a multiple-item scale for assessing electronic service quality," Journal of Retailing, vol. 64, no. 1, pp. 12-40, 2005.

[8] G. R. Heim and K. K. Sinha, "Operational Drivers of Customer Loyalty in Electronic Retailing: An Empirical

The 6th International Conference on Cyber and IT Service Management (CITSM 2018) Inna Parapat Hotel – Medan, August 7-9, 2018

Analysis of Electronic Food Retailers," Manufacturing and Service Operation Management, vol. 3, no. 3, 2001.

[9] S. Cai and M. Jun, "Internet users' perceptions of online service quality: a comparison of online buyers and information searchers," Managing Service Quality: An International Journal, vol. 13, no. 6, pp. 504-519, 2003.

[10] C. H. Yen and H. P. Lu, "Effects of e-Service Quality on loyalty intention:an empirical study in online auction," Managing Service Quality:An International Journal, vol. 18, no. 2, 2008.

[11] W. J. Doll and G. Torkzadeh, "The Measurement of End User Computing Satisfaction," MIS Quarterly, vol. 12, no. 2, pp. 259-274, 1988.

[12] Statista, "U.S. customer satisfaction with eBay Inc. from 2000 to 2017," Statista, 2018. [Online]. Available: https://www.statista.com/topics/2181/ebay/. [Accessed 20 2 2018].

[13] D. Easley and J. Kleinberg, Networks, Crowds, and Markets: Reasoning about a Highly Connected World, 2010.

[14] eBay, "2016 Annual Report," 2016. [Online]. Available: http://files.shareholder.com/downloads/ebay/0x0x935618 /61DC48DD-11A4-437A-81A7- 39F3FE892B68/Final_eBay_2016_Annual_Report.pdf. [Accessed 18 02 2018].

[15] R. Copeland and N. Crespi, "Analyzing consumerization - should enterprise Business Context determine policy decision?," IEEE ICIN, 2012.

[16] C. N. Madu and A. A. Madu, "Dimensions of E-Quality," International Journal of Quality and Reliability Management, vol. 19, 2002.

[17] A. Molla and Licker, "E-commerce Systems Success: An attempt to extend and respecify the Delone and MacLean Model of IS Success," Journal of electronic commerce research, 2001.

[18] K. Waite and T. Harrison, "A Consumer Expectations of Online Information Provided by Bank Websites," Journal of Financial Service Marketing, 2002.

[19] H. Teo, I. Oh, C. Liu and K. K. Wei, "An Empirical Study of Effects of Interactvitity on Web user attitude," International Journal of Human Computer Studies, 2003.

[20] Delone and E. McLean, "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research, vol. 3, no. 1, pp. 60-95, 1992.

[21] T. Dyke, L. Kappelman and V. Prybutok, "Measuring Information Systems Service Quality: Concerns on the Use of the SERVQUAL Questionnaire," MIS Quarterly, vol. 21, no. 2, pp. 195-208, 1997.

[22] M. Isman, "Measuring Information Success at the Individual Level in Cross-cultural Environments," Information Resources Management Journal, vol. 9, no. 4, pp. 16-28, 1996.

[23] V. Zeithaml, "Service Quality, profitability and the economic worth of customers: what we know and what we need to learn," Journal of the Academy of Marketing Science, 2000.

[24] S. Kim and L. Lim, "Consumers’ Perceived Importance of and Satisfaction with Internet Shopping," Electronic Markets, vol. 1, 2001.

[25] D. Halstead, D. Hartman and S. Schmidt, "Multisource Effects on the Satisfaction Formation Process," Journal of the Academy of Marketing Science, 1994.

[26] R. Oliver, "Processing of the Satisfaction Response in Consumption: A Suggested Framework and Resarch Propsotiions," Journal of Marketing Research, 1989.

[27] B. Liu, "Sentiment Analysis and Opinion Mining," Morgan & Claypool Publishers, 2012.

[28] B. Pang and L. Lee, "Opinion Mining and Sentiment Analysis," Foundations and Trends in Information Retrieval, vol. 2, pp. 1-135, 2008.

[29] E. Turban, J. Aronson and T. Liang, Decision Support Systems and Intelligent Systems 7th ed., Prentice-Hall, 2005.

[30] A. Harb, M. Planti, G. Dray, M. Roche, O. Fran, Trousset and P. Poncelet, Web Opinion Mining: How to Extract Ppinions from Blogs?, 2008.

[31] W. Quesenbery, What Does Usability Mean: Looking beyond ‘Ease of Use, ociety for Technical Conference, 2001.

[32] F. Reichheld, "Loyalty and the Renaissance of Marketing," Marketing Management, vol. 2, no. 4, pp. 10- 21, 1995.

[33] T. Thurau and A. Klee, "Impact of Customer Satisfaction and Relationship quality on Customer Retention: A Critical Reassessment and Model Development," 1997.

[34] V. Pujani, "Use of Ecommerce Website in Developing Countries," International Journal of Economics and Management Engineering , vol. 5, no. 6, 2011.

[35] F. Mohsan, M. M. Nawaz, M. S. Kham, Z. Shaukat and N. Aslam, "Impact of Customer Satisfaction on Customer Loyalty and Intention to Switch: Evidence from Banking Sector of Pakistan," International Journal of Business and Social Science, vol. 2, no. 16, 2011.