help

978-1-5386-3592-6/17/$31.00 ©2017 IEEE

Sentiment Analysis of Innovations in Russian Media

Khramoin I.V., Kamenskaya M.A., Tikhomirov I.A. Institute for Systems Analysis

Federal Research Center “Computer Science and Control” of Russian Academy of Sciences

Moscow, Russia [email protected], [email protected], [email protected]

Toganova N.V. Primakov National Research Institute

of World Economy and International Relations Russian Academy of Sciences

Moscow, Russia [email protected]

Abstract—The paper presents the sentiment analysis based algorithm for evaluation of public's attitude to innovations. Domain-depended corpus for training algorithm was created. More than 200 000 articles from Russian mass media devoted to innovations were analyzed. Results of analysis represented as set of histogram. Conclusions and future work are given.

Keywords—sentiment analysis; analysis of mass media; innovative development; technology development

I. INTRODUCTION The attitude of population to innovative development is

one of the factors influencing technological development. The decision makers’ attention to public opinion demonstrates its importance. Public opinion polls are regularly taking place in the European Union (Eurobarometer Special Issues) and section of the Public Understanding of Science and Technology is always presented in the US National Science Foundation annual.

There are suggestions that people's attitudes toward a process are affected by their awareness. Many researchers consider the relationship between awareness and attitude as an axiom. As a rule it is considered the more information an individual has the more positive his attitude. Within the theory of diffusion of innovations the following ways of spreading information are considered: mass media, social networks or personal communication [1]. Although the latter two receive more attention [2, 3], but the media keep the influence: the case-study showed that they continued to play an important role in the dissemination of some innovations. Research has shown that the media influenced the medical sector [4], politicians [5], the public's perception of nuclear [6] and green energy [7].

In this paper we consider two questions. First we wanted to answer questions about the awareness of the community about the innovation process and the influence of the media on the society attitude towards the ongoing processes. Secondly we wanted to develop a sentiment analysis method adapted for the analysis of the media. Sentiment analysis method consists in the search for positive or negative references to technology or innovation. There are a lot of different sentiment analysis methods nowadays. However, the application of sentiment analysis to media research required some adjustments to

existing methodologies because, unlike social networks or user feedback, which are often subject of sentiment analysis, media articles are presented in a more consistent style, making it difficult to use common emotive patterns for analysis.

The first section of the article describes the initial data used in the study. The second section presents developed method for sentiment analysis of news media. The third section is devoted to the experimental results and their discussion.

II. DATASETS At the first stage we selected media sources which we are

going to analyze. The list of 16 news feeds was used in our experiments. We selected more than 240 thousands articles dedicated to innovations and technologies which were published in the period from 2005 to 2015. Only innovations and technologies related to “The list of Critical technologies of the Russian Federation” [8] were considered. These articles were automatically indexed by Exactus Expert [9].

When articles have been collected they were categorized into 11 technological trends (topics) from “The list of critical technologies of the Russian Federation”. We manually chose keywords for each topic and used them to categorize selected articles. For example, biotechnology keywords are: bioengineering, biomedicine, nano medicine, bio pharmacology, bioinformatics, bionics, bioremediation, artificial selection, cloning, hybridization, genetic engineering, transgenic plants, transgenic animals. The quantitative distribution of mentions by category (see Table 1) generally corresponds to the intuitive idea of what excites society, which areas of technology are relevant and promising.

Some of the collected articles were manually marked to create the training sample. The pairs “the object of tonality + an emotive construction” were marked. About 120 articles containing technologies mentioning in a positive or negative context were selected from the existing collection of news articles to create a marked corpus. In the process of marking the mention of technologies was singled out as an object of tonality and the context as an emotive construction. At the same time each pair received a property – modality sign, indicating the exact context, the technology is mentioned. For example, in the following fragment the phrase “cloud technologies” will be selected as the object of tonality, and “they will be able to build an effective multi-level interaction” is an emotive construction, and the modality sign is “positive”: The research was supported by Russian Foundation for Basic Research, Grant № 14-29-05078.

96

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:32 UTC from IEEE Xplore. Restrictions apply.

TABLE I. TOPICS AND ARTICLES

Topic Number of article 1 Information Technology 19 501 2 Healthcare 18 293 3 Nanotechnology 17 306 4 Power Engineering 16 473 5 Environment 16 013 6 Minerals 12 084 7 Transport 9 201 8 Military technology 7 146 9 Emergencies 5 385 10 Energy Efficiency 1 944 11 Biotechnology 1 517

“Progress does not stand still, and a growing number of our customers are choosing cloud technologies. Our technology makes it possible to build effective multi-level interaction between management and employees as part of today's mobile society.”

Thus, it was marked out 346 pairs: 272 pairs belong to positive cases mentioned technologies and a 74 pairs belongs to negative mentions. The marked corpus analysis showed that the experts favored a long emotive constructions (consisting of several words, 3 or more). Emotive constructions’ elements have been associated syntax. However the chance to meet a complete coincidence allocated structures is low because of the paraphrase. Therefore in our research long emotive constructions converted to the form: the key word + its dependent words. Also the similarity criterion was introduced: two constructions will be identical if their main words are the same as well as at least 70% of relative words of the constructions intersects.

III. METHODOLOGY As mentioned, sentiment analysis of text fragments

dedicated to technologies and innovations is the main aim of our research. The point of sentiment analysis is detecting text fragments, named emotive constructions, which transmit the relation of the subject to the object of statements.

A preliminary analysis of the mass media has shown that a mature style of material presentation is generally accepted in the journalistic style and the opinion is expressed by terms specific to the object of the statement, therefore, the procedure of the additional learning of the sentiment analyzer requires [10]. For this purpose a training collection was used.

At the stage of training we first find the constructions, experts pointed out, and categorize them into predefined topics. Then, we find emotive constructions, specific to a category, and general emotive constructions. An analysis of the position of the construction in the text with respect to the tonality object was performed to assign each found construction to the lexicon of a particular category or general emotive vocabulary on the basis of the syntax tree. We counted trust rating that shows the probability that the found emotive construction points to a certain modality sign.

A. Learning Algorithm A detailed learning algorithm is given below.

Step 1. Search in the text of the constructions that were assigned by experts to the categories of the objects of tonality.

Step 2. If the object of tonality is found, then:

The marked constructions are added to the list of emotive constructions specific to the category received. The modality sign and the position type of this construction with respect to the object of tonality based on the syntax tree is pointed out. There are 4 position types:

i. Emotive construction belongs to the descendants of object of tonality

ii. Emotive construction belongs to the parents of the object of tonality

iii. Emotive construction is higher and connected through the verb

iv. Emotive construction is not syntactically associated with the object of tonality (they are in different sentences)

If the current emotive construction’s object of tonality doesn’t correlate with the text category, this construction will be inserted into list [x].

Step 3. The marked construction falls into the list of hypothetical general emotive constructions together with the modality sign.

Step 4. All texts are analyzed:

General emotive constructions that respond to several modality sings from the list of hypothetical general emotive constructions are filtered out. Trust raiting for remaining constructions is counted with (1):

c

i ic docs

ent S ,

where Sc,i – an estimate of the general emotive construction i belonging to the class c,

enti – the number of mentions of construction i,

docsc – number of texts, related to the modality sign c.

Emotive constructions of each category is evaluated. The score is calculated for each modality sign with the formula (2):

N N

EW cc

where с – modality sign,

Nc – the total number of occurrences of the emotive construction related to the modality sign c,

97

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:32 UTC from IEEE Xplore. Restrictions apply.

N – the total number of occurrences of the emotive construction.

The constructions from the list [x] are estimated for each category with (3):

ymymt

mtt ymyt

ym

ymymt

ym vv vvclos

v vv

CatTrust ,,

,1 ,,

,

,,

, )max(, ),(1

)max(,0

where m – number of the category of the construction under consideration,

t – category index, skipping the considered category,

y – considered construction,

vm,y – the frequency of occurrence of the construction y in the category m.

ymyt

ymyt ymyt vv

vv vvclos

,,

,, ,, ,1

,0 ),(

As a result of the training a list of emotive constructions is obtained, in which each word corresponds to a positive or negative modality sign with the estimate of this membership. As an example, we cite fragments of the resulting dictionaries of positive and negative emotive vocabulary for the category “Information Technology” (see Table 2).

TABLE II. EMOTIVE VOCABULARIES

Positive vocabulary Negative vocabulary

высокая производительность (high performance)

лучшее решение (the best decision) простота изменений (ease of change) хорошие перспективы

(good prospects) удостоиться премии (to win a prize) выдающиеся характеристики

(outstanding performance) повысить качество

(to improve the quality) снизить риски (to reduce risks) превосходить других

(to outperform others) повышать надежность

(to improve reliability)

сомнительное качество (dubious quality)

не отвечать стандартам (does not meet the standards)

ниже уровня (to be below) шаг назад (step back) дороже (expensive) иметь недостаток

(to have the disadvantage) выглядеть смешно (to look funny) не могут сравниться

(cannot be compared) высокая зависимость

(high dependence) не оправдать (do not justify) малая эффективность

(low efficiency)

B. Classification algorithm Classification algorithm is divided into two stages: the

search for the object of tonality belonging to one of the specified categories and decision about the modality sign of the statement according to the object of tonality.

We define the category following these steps:

Searching for structures belonging to one of the categories is in progress. If such a construction is found, then the category is defined, the stage will be completed. Otherwise, proceed to point 2).

Searching for structures from the list [x]. If such constructions are found and they all belong to the same category, then the category will be defined. The stage is completed. If no constructions are found, the text won’t fall into any category, the stage is completed. Otherwise, proceed to item 3).

Coefficients of words found in the list [x] are summed for each category. On the basis of these values the category that has received the maximum number is selected. If there are several categories, the category, the maximum ratio construction belongs to the only category, will be chosen.

Assignment to the tonality class is performed on the basis of a trust ratio. Trust ratio (TrustVal) reflects the probability of found emotive constructions point to a certain modality sign.

The definition of the modality sign is carried out in stages. At the first stage we deal with the emotive constructions specific to the identified category:

Search for emotive constructions specific to the identified category.

Emotive constructions, which were pointed out at the previous step, are compared with the objects of tonality and words from the list [x], the trust ratio of relation to the modality level is used (5):

n

i i

ic c PosKoef

EW TrustVal

1

,

2

where c – modality sign,

n – number of pointed out emotive constructions,

EWc,i – the score of connection between emotive construction i and modality sign c

PosKoefi – indicator of finding the object of tonality or the list [x] constructions with the position type corresponding to the one identified in the learning phase. 0 – if not found, 1 – in the opposite case.

The modality sign with the maximum TrustValc is selected. If we find the only modality sign with the highest TrustValc then the received modality sign is considered the answer. Otherwise, go to the second stage.

The second step responds to evaluation of modality signs with the same TrustValc highest value using the general emotive constructions allocated at the previous stage. We are trying to find general emotive constructions for each considerable modality sign. Using the search result we calculate new trust ratio with (6):

icc STrustVal ,

We choose modality level with the highest trust ratio. If we find more than one modality level again, the modality level cannot be found.

98

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:32 UTC from IEEE Xplore. Restrictions apply.

IV. EXPERIMENT AND RESULTS During the research distributions of positive and negative

mentions of technologies and innovations in the media were received (see Fig. 1). Further expert analysis of the obtained data has shown that they often write neutral about technologies in the media. This can be explained by the consistency of the style of news materials. The distribution on the basis of the context – “praise” or “scold” – for the whole set of articles over the period under review showed that the share of negative feedback on military technologies is quite expectedly higher. At the same time the comparatively low share of mentions in the negative context of information technologies and the positive attitude of these articles in general shows that society is not concerned with the negative consequences of these technologies (for example, violation of personal space, disclosure of personal information, espionage, etc.). The result of biotechnology shows the same: the ethical complexities of the development are not discussed on the pages of the media, while technologies for protecting the environment or necessary for the prevention of emergencies are obviously more diverse (they are mentioned, including in negative context).

Fig. 1. Mentions distributions for technologies (average value 2005-2015).

The same way we can provide sentiment analysis of technology mentions on years. When analyzing the mention of technology in a breakdown by years the situation changes. Consider the mentions’ rating for information technology. As noted above, news about information technology is mainly presented in a positive way, but when analyzing the emotional coloring of articles distribution by year, it is obvious: information technologies are gaining popularity (see Fig.2).

Fig. 2. Rating of positive and negative mentions for IT.

It seems quite reasonable: after all, information technologies play an increasing role in all sectors of the economy; they are often written about in connection with the industrial Internet and related technologies. On the contrary, analysis of

distribution results for other technologies shows that, despite the growth in the number of articles during the decade under review, there has been no change in attitude towards other technologies during this time.

V. CONCLUSION In this paper a method for sentiment analysis of

technologies and innovations mentions in the mass media was proposed. The method takes into account the peculiarities of the vocabulary used in news texts. During the research the training corpora was annotated. After training the analyzer experiments were conducted on the datasets of articles about technologies and innovations. The conducted research showed that sentiment analysis can be used as a tool for analyzing innovative development to reduce the laboriousness of the expert performing this analysis.

The complex quantitative and qualitative study of the media as a reflection of the public's notion of innovation, and also as a channel that influences these ideas, has shown that society is more interested in this subject in the last decade – there has been an increase in the number of publications devoted to innovations. At the same time the number of publications devoted to specific technologies shows special attention to those that are actively supported by government programs. Thus the attention of society is shifting towards concrete, not always promising directions.

REFERENCES [1] N. Lin, R.S. Burt, “Differential Effects of Information Channels in the

Process of Innovation Diffusion”, Social Forces. Vol. 54. No 1. 1975. pp. 256-274.

[2] P.S. van Eck, W. Jager, P.S.H. Leeflang, “Opinion Leaders’ Role in Innovation Diffusion: A Simulation Study”, Journal of Product Innovation Management. Vol. 28, No 2. 2011. pp. 187-203.

[3] T.W. Valente, R.L. Davis, “Accelerating the Diffusion of Innovations Using Opinion Leaders”. The ANNALS of the American Academy of Political and Social Science. Vol. 566. No 1. 1999. pp. 55-67.

[4] M. Haider, G.L. Kreps, “Forty Years of Diffusion of Innovations: Utility and Value in Public Health”, Journal of Health Communication. Vol. 9 (sup1). 2004. pp. 3-11.

[5] M.H. Kepplinger, “Reciprocal Effects: Toward a Theory of Mass Media Effects on Decision Makers”, The Harvard International Journal of Press/Politics. Vol. 12. No 2. 2007. pp. 3-23.

[6] W.A. Gamson, A. Modigliani, “Media Discourse and Public Opinion on Nuclear Power: A Constructionist Approach”, American Journal of Sociology. Vol. 95. No 1. 1989. pp. 1-37.

[7] R. Wüstenhagen, M. Wolsink, M.J. Bürer, “Social acceptance of renewable energy innovation: An introduction to the concept”, Energy Policy. Vol. 35. No 5. 2007. pp. 2683-2691.

[8] List of critical technologies of the Russian Federation. Official site of the President of the Russian Federation. http://www.kremlin.ru/supplement/988 Cited: Sept 23, 2017.

[9] G. Osipov, I. Smirnov, I. Tikhomirov, I. Sochenkov, A. Shelmanov, “Exactus Expert – Search and Analytical Engine for Research and Development Support”, Novel Applications of Intelligent Systems. Springer International Publishing, 2016. pp. 269-285..

[10] I.V. Khramoin, “Sentiment analysis method with the use of word combinations”, Proceedings of the third All-Russian scientific conference of young scientists with international participation "Theory and practice of system analysis." Volume II. Rybinsk, 2014. pp. 171-175 (in Russian).

99

Authorized licensed use limited to: University of the Cumberlands. Downloaded on October 17,2021 at 14:54:32 UTC from IEEE Xplore. Restrictions apply.