Reading assignment for Time and Change

Player Profiling with Fallout 3

Pieter Spronck, Iris Balemans, and Giel van Lankveld Tilburg Center for Creative Computing

Tilburg University, The Netherlands

Abstract

In previous research we concluded that a personality profile, based on the Five Factor Model, can be constructed from ob- servations of a player’s behavior in a module that we designed for Neverwinter Nights (Lankveld et al. 2011a). In the present research, we investigate whether we can do the same thing in an actual modern commercial video game, in this case the game Fallout 3. We stored automatic observations on 36 participants who played the introductory stages of Fallout 3. We then correlated these observations with the participants’ personality profiles, expressed by values for five personality traits as measured by the standard NEO-FFI questionnaire. Our analysis shows correlations between all five personality traits and the game observations. These results validate and generalize the results from our previous research (Lankveld et al. 2011a). We may conclude that Fallout 3, and by ex- tension other modern video games, allows players to express their personality, and can therefore be used to create person- ality profiles.

Introduction People tend to ascribe personality traits to each other, which are assumed to be stable over time and across situations. Personality theory is concerned with such stable patterns in people’s behavior (McCrae and Costa 1997). Behavioral psychologists model personality as a set of traits, which are thought to be biological drives that cause major categories of behavior. The personality model that we focus on in this paper is the Five Factor Model of personality, also known as the “Big Five.” This model has emerged as the most widely accepted for the purpose of modeling personality (Digman 1990).

Gathering data to score individuals on the Five Factor Model is generally done using personality questionnaires, despite the fact that data gathered through interviewing or direct observations is considered to be more reliable (Mount, Barrick, and Strauss 1994). However, the need for extensive and wide data gathering makes interviewing and observing time-consuming and expensive. Questionnaires provide a reasonable alternative, even though their advantage in time requirements comes with a decrease in reliability (Kolar, Funder, and Colvin 1996).

Copyright c© 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Differences between players lead to different playing styles, decision making, and preferences. Assuming that there are stable patterns in game behavior and assuming that these patterns are related to personality, games may be used as an alternative for questionnaires for establishing person- ality profiles. In this research we investigate whether in- dividual differences in video game behavior are related to differences in personality, as expressed by the Five Factor Model.

We investigate whether personality is present in game behavior by correlating recorded game behavior to scores on the NEO-FFI personality questionnaire. In previous research (Lankveld et al. 2011a), we used a Neverwinter Nights module that we designed ourselves, called The Poisoned Lake, for recording game behavior. In the present research, we repeat our previous work, but we use the com- mercial video game Fallout 3 (Bethesda Softworks 2008). Besides investigating to what extent correlations between personality and game behavior exist, we also compare the results achieved with Fallout 3 with the results we achieved with The Poisoned Lake.

Background In game research, several researchers have investigated the relationship between a player’s profile and his appreciation of game contents. The common approach is to define player profiles based on gaming archetypes. Typical examples are Thue et al.’s (2007) PaSSAGE, Canossa’s (2009) play- persona’s, Yee’s (2006) player motivations, and Bateman and Boon’s (2005) DGD1 play styles. In contrast, in our work we aim to base our player profiles on well-grounded psychological personality theory.

According to Costa and McCrae (1995), personality is defined as the stable patterns of variation in individual act- ing, thinking, and experiencing. Personality arises from in- teractions between (1) the situation in which the individual is placed, and (2) processes that take place in the individual (Back and Egloff 2009). Personality theory implicitly as- sumes that personality results from interactions. However, personality scores are a result of measurements across situ- ations and can therefore be generalized (Ten Berge and De Raad 2002).

In this research we focus on the Five Factor Model of per- sonality (FFM). This model claims that by using five per-

179

Proceedings, The Eighth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment

sonality traits all variations of personality across all ages, sexes, and cultures can be described. Already in 1934, Thurstone suggested that five personality domains were suf- ficient to describe a person’s personality. His suggestion was supported by numerous other researchers, which led to the Five Factor Model (Wiggins 1996). The model as used nowadays was designed by analyzing the natural-language terminology that people use to describe one another (John and Srivastava 1999).

The five personality traits used in the Five Factor Model are (1) openness to new experiences, (2) conscientiousness, (3) extraversion, (4) agreeableness, and (5) neuroticism (McCrae and Costa 1997). This set of five traits is often abbreviated to “OCEAN.” Trait scores follow a normal dis- tribution. Below we give a description per trait in terms of behavior that can be seen in natural human settings. The exact definition of a trait tends to vary slightly between re- searchers. Here we adhere to the common definitions.

• Openness: interest in novel stimuli. A high score is typic- ally accompanied by curiosity and willingness to deviate from social conventions.

• Conscientiousness: propensity to adhere to social and personal rules. This trait is also tied to the ability to re- strain oneself and the ability to stick to a plan during peri- ods of stress and difficulty.

• Extraversion: seeking excitement and positive stimuli. High scorers seek the company of others and exhilarat- ing situations like high speed driving, rollercoasters and other high-adrenaline activities.

• Agreeableness: compliance, willingness to cooperate, and friendliness. High scorers are seen as empathic, while low scorers put their own needs above those of others.

• Neuroticism: propensity to experience fluctuating and negative emotions such as anger and fear. High scorers are more likely to be aware of risks and safety. There is also a relationship to shyness and social anxiety.

Commonly, personality is measured using personality questionnaires. These questionnaires contain a list of de- scriptive statements for which the participant has to indic- ate the measure in which the statements describe him or her. Questionnaires are typically designed using factor ana- lysis to cluster large lists of descriptive terms (McCrae and Costa 1997). Scores on personality questionnaires correl- ate strongly with various areas of human behavior and pref- erence. Examples are job preference, relationship styles (White, Hendrick, and Hendrick 2004), and smoking beha- vior.

The NEO-PI-R questionnaire, developed by Costa and McCrae (1995), is a widely accepted test for measuring per- sonality according to the Five Factor Model. In the present research we use the NEO-FFI questionnaire, which is a shortened version of the NEO-PI-R. The NEO-FFI is less reliable than the NEO-PI-R on the level of the 30 facets un- derlying the five personality traits, but there is no signific- ant difference between the two questionnaires on the level of personality traits themselves. As in this research we are

only interested in the five personality traits, we opted for us- ing the NEO-FFI to save about half-an-hour per test subject.

Experimental Setup In our research we investigated to what extent personality scores correlate with game behavior. In order to perform our experiment we applied two measurements: (1) participants took the NEO-FFI personality test, and (2) the same parti- cipants played Fallout 3, while we recorded their in-game behavior. Below we supply general information on the par- ticipants and the experimental procedure, the game we used, and the variables that we used to measure behavior in the game.

Participants and procedure We recruited 36 participants for our experiment from the stu- dent population. Their ages ranged from 18 to 27. 18 par- ticipant were male, 17 were female, and one did not report gender. Only students who had not played Fallout 3 before were allowed to participate.

All participants played the game under the same condi- tions: in the same room, at the same computer, in the same timeframe. The room was an office, tidied up as much as possible to ensure few distractions. The main researcher was in the room, but 3 to 4 meters away from the participant, and clearly in a position unable to view what the participant was doing.

At the start of the experiment, the participant received a list of written instructions to the game. These instructions described the game controls, and gave a general indication of what the participant was supposed to do in the few sec- tions of the game that were used. The participants were al- lowed to consult these instructions during the experiment. Furthermore, during the first two phases of the game, which were training areas, they were allowed to ask the researcher questions. For the rest of the game, they were not allowed to do that.

The participants were allowed 45 minutes to play the game. In that time they had to play through four phases of the game. After the fourth phase, a message would pop up instructing the participant to stop playing. Participants who were not yet finished with the game after 45 minutes, were interrupted by the researcher. Most participants had sufficient time to finish the game, and the four that were in- terrupted by the researcher after 45 minutes were actually close to finishing.

After playing the game, the participants completed two questionnaires: the NEO-FFI, and a short demographics questionnaire. All data gathered on the participants was re- corded anonymously.

Game Fallout 3 is a first-person game that combines shooter ele- ments with RPG elements. Its background is a world rav- aged by nuclear war, in which part of the population has fled into underground “vaults” to escape the horrors of the sur- face. The protagonist of the story, controlled by the player,

180

Figure 1: Conversation in Fallout 3.

has lived his (or her) life in one of those vaults until reach- ing 19 years of age. After that he escapes the vault and gets confronted with the remains of civilization on the overland.

The game starts with a several training phases, in which the player gets to create his in-game persona, learns how to control the game through the mouse and keyboard, and how to interact with the game world. These training phases all take place inside the vault. During our experiment, the participants played the first four phases of the game: (1) the birth of the player character, (2) being 1 year old, (3) the player’s 10th birthday party, and (4) taking an aptitude test when 16 years old.

We chose the game Fallout 3 for our experiment for five reasons: (1) it is a popular, recently-released, com- mercial video game; (2) it uses a first-person perspective and realistic character models, which stimulate immersion; (3) it allows the expression of a variety of player prefer- ences through conversation, action, and unconscious beha- vior; (4) its opening sequence, used in the experiment, is mostly story-driven with little violence; and (5) a toolset for the game has been released that allowed us to change the game to automatically record player decisions and actions.

The game uses a standard control interface: the W, A, S, and D keys are used for moving, and the mouse is used for looking around and fighting. There are a few more control possibilities, but none of these were needed during the ex- periment. Moreover, whenever they were applicable, they were listed on the screen. In-game conversation is per- formed using a conversation menu (see Figure 1): when talk- ing to another character, the player gets to select an answer from a list of 1 to 6 possible answers. The choices made dur- ing conversations may have an impact on the remainder of the conversation, and on situations occurring after the con- versation is over.

Measured variables In this experiment we collected data of the participants’ in- game behavior. We describe two categories of variables: (1) unpooled, which are directly measured in the game, and (2) pooled, which are the sum of several unpooled variables. All variables in the game are natural numbers with unlim-

Figure 2: Six triggers are placed in this environment.

ited range (though in practice the largest value that we en- countered was 28).

Unpooled variables We collected values for 165 in-game variables of four different types. We called these the “un- pooled” variables, to contrast them with the “pooled” vari- ables (which are explained below). The four different types are: (1) conversation, (2) movement, (3) G.O.A.T., and (4) fight.

Conversation variables were created for each possible conversation choice that the player could make, and coun- ted how often the corresponding choice was made. Usually these could only get values zero and 1. Figure 1 shows a typical example of a conversation with three choices.

Movement variables corresponded to triggers that we placed in the environment. The variables counted how of- ten a player moved through its corresponding trigger. We placed triggers both in locations where the player needed to go to advance the game, and also in places that the player could visit optionally. Figure 2 shows a typical example for triggers placed, in this case during stage 3 of the game (the 10th birthday party). Only 2.3 and 2.4 need to be activated by the player, and each only once; the other locations are visited optionally.

G.O.A.T. variables corresponded to choices that the player made during the final stage of the game, where an aptitude test, called “G.O.A.T.”, is taken. We separated these from the conversation choices, as most of the answers that the player can supply on the test are rather whimsical. We felt that we could not learn much from the player’s choices here.

The fight variable is a single variable that checked whether or not the player entered into a fight with the “Tun- nel Snakes,” a local gang that terrorizes the hallways of the Vault.

Pooled variables Our previous research showed that per- sonality effects of conversation or movement might be found that only appear when examining average effects across en- tire areas of game behavior. We therefore created 27 pooled variables that collect the values of all unique variables per

181

area, per NPC, and for the entire game. These pooled vari- ables can, for example, be used to examine the overall tend- ency of a player to move around or to engage in conversa- tion. Individual unpooled variables might miss such tend- encies. In pooling we counted and summed the values of the variables to form the pooled variable. The following list presents the overview of the final set of pooled variables that we used.

• Group 1 contains 2 pooled overall variables: one which sums all conversation variables in the game, and one which sums all movement variables in the game.

• Group 2 contains 7 pooled movement variables. Move- ment triggers were placed in stage 3 and stage 4 of the game; stage 3 containing 3 sub-areas, and stage 4 con- taining 2 sub-areas. The pooled movement variables sum the values of all triggers for each of the sub-areas, and for the wholes of stage 3 and stage 4.

• Group 3 contains 2 pooled conversation variables: one that sums all conversation variables for stage 3, and one for stage 4.

• Group 4 contains 16 pooled conversation variables, namely one for each NPC which can be conversed with in the game. An NPC that is encountered in stage 3 and also in stage 4 is considered to be two separate NPCs.

A fifth group was added to make the variable set complete. This group contains the 165 unpooled variables.

Questionnaires For the current research two questionnaires were used. The first questionnaire was the NEO-FFI. The second question- naire was based on a questionnaire used by Schreurs (2009), when performing research similar to ours. This question- naire concerned participant demographics. It asked about the particpant’s gender, age, education level, and experience with computers, computer games, and the English language.

Both questionnaires were completed by the participants after playing the game. Because the study of Schreurs (2009) showed that there were no order effects concerning whether the participants started with the questionnaires or the game, in the current research we did not take order ef- fects into account.

Results The outcomes of the questions per trait were totalled, and then normalized into sta-nine scores (scores from 1 to 9) conforming to the norms table in Hoekstra, Ormel, and De Fruyt (2007) used for participants in scientific research. Table 1 shows the descriptive statistics of the scores. These indicate that our participant pool are close to population av- erages.

We performed a linear regression analysis in order to see if the OCEAN traits could be predicted using the pooled and unpooled game variables. The results are found in Table 2. Column 1 contains the OCEAN personality traits, column 2 contains the effect size (R2), and column 3 contains the number of game variables used in the linear model. The

Table 1: Descriptive statistics of NEO-FFI scores. Minimum Maximum M SD

Openness 3 9 5.81 1.60 Conscientiousness 1 7 4.03 1.54 Extraversion 2 9 5.86 1.87 Agreeableness 1 8 4.17 1.99 Neuroticism 1 7 4.53 1.63

Table 2: Stepwise linear regression analysis. Trait R2 Variables in model Openness .982 15 Conscientiousness .486 4 Extraversion .837 8 Agreeableness .752 6 Neuroticism 1.000 23

effect size represents the so-called “goodness of fit” of the model. “Goodness of fit” denotes the amount of variance explained by the model. Effect sizes range from zero to 1, representing the fraction of the variance that is explained by the model (e.g., an effect size of 0.5 indicates that 50% of the variance is explained). The table shows that we are cap- able of predicting both Openness and Neuroticism with a certainty above 95%, and that we are also capable of predict- ing Extraversion and Agreeableness with a high certainty.

In order to examine the results more closely, the 1-tailed correlations corresponding to the linear regression analysis were examined in detail. Table 3 contains the total number of positive and negative significant correlations per group of variables. We included only significant correlations in the table; an empty cell means there was no significant correl- ation. As can be seen in Table 3, significant correlations were found for every group of game variables. Each of the 5 personality traits had at least one correlating variable. This means that significant effects are present both in conversa- tion and in movement behavior in the game. A total of 87 of the 192 game variables showed significant correlations with one or more personality traits (some of them correlated with multiple traits). This is 45% percent of all the game vari- ables.

Discussion We examined the relationship between personality and game behavior in the commercial video game Fallout 3. For all five traits of the Five Factor Model, significant correlations with in-game behavior were found. Below, the present study is compared to our previous study using the Poisoned Lake game (Lankveld et al. 2011a). Furthermore, we make some general remarks on personality in games, and make an at- tempt to interpret the results.

Comparison with previous research While both the present study and our previous study (Lank- veld et al. 2011a) had similar designs and similar goals, two main differences between the two studies should be noted:

182

Table 3: Positive and negative correlations per variable group. Group 1 Group 2 Group 3 Group 4 Group 5 Total

Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Pos Neg Openness 1 12 7 13 7 Conscientiousness 1 3 9 4 10 Extraversion 1 16 6 17 7 Agreeableness 1 3 1 4 11 5 15 Neuroticism 2 1 3 1 16 1 22

(1) the present study contained variables that were differ- ently set up and fewer in number, and (2) the contents of the games used were rather different.

(Ad 1) The variables used in current research and in the study of the previous chapter are different in two ways: (a) in the previous study we used a self-created game with con- versation variables that were designed to represent one of the personality traits, which is not the case in present study; and (b) in the previous study movement variables were placed on locations where the player comes often and locations where the player does not come often, while in the present study movement triggers were mainly placed where players in general do not come often.

(Ad 2) The contents of the Poisoned Lake game of the previous study were designed to allow players to exhibit be- havior that was in some way related to specific personal- ity traits, while no such design is underlying Fallout 3. A clear example is that conversation in the previous study is set up to provide variety in the possible answers while the present study contains conversations with some characters in which most of the responses are denials and sarcastic com- ments. Another example is that the game in the previous study contained several optional side-quests that were de- signed to show off specific personality traits, such as warn- ing NPCs of approaching danger. Such side-quests were ab- sent in the present study. The visuals of the two games were also remarkably different: The Poisoned Lake uses a 3rd- person view and has a medieval setting, while Fallout 3 uses a 1st-person view in a modern setting.

In the previous study we noted that almost 70% of the variables correlated in some way to personality traits; in the present study that number is much lower, namely 45%. The differences in design of the games might be an explanation for that lower number.

Expressing personality in games To our knowledge there is no research comparing the be- havioral differences between a game and real life. Our as- sumption is that those differences do exist, but that the be- haviors concerned are still related. For example, when the player swears at an NPC in a game, that does not mean that he would do such a thing in real life. However, it still in- dicates that the player finds enjoyment in swearing, or that he does not think that it is a problem within a certain game situation. Even when someone behaves differently in a game than in ‘real life’, personality influences their in-game beha- vior. If that assumption holds, we can expect that ‘real-life’ personality is expressed by in-game behavior in some form,

which makes our approach viable. Still, player behavior is only partially driven by personality – we found that general experience with games is highly influential in determining how a player expresses himself in a game.

One problem we encountered was how to prevent par- ticipants from choosing answers randomly instead of con- sidering their choices carefully. In the study by Schreurs (2009), participants received the instruction to behave in the game as they would behave in reality. In present research we chose not to do this, because such an instruction is intro- spective, unrealistic (because the situations within the game are not realistic), and unnecessary (because personality is always involved). Instead, we explained in the instructions that the participants have to play the game as if they were going to finish it. In this way we tried to strengthen the ecological validity. The participants knew that their choices were important in the context of the game, because they de- termined the further proceedings of the game.

Interpretation of the results When examining which player behaviors correlate highly with personality traits, we see the following: (1) High openness corresponds to exploring the locations by moving around; (2) High conscientiousness corresponds to avoid- ing confrontational conversation options; (3) High extraver- sion corresponds to picking friendly, social, and affective conversation choices; (4) High agreeableness corresponds to following the instructions of the game rigidly (moving only where necessary), and talking to authority figures; and (5) High neuroticism corresponds to avoiding conversations and taking long to finish the game.

The question is: can we generalize over these observa- tions based on the present experiment? The answer is: prob- ably not. The reason is that due to the relatively low number of participants, and the high number of features, the correl- ations that we found are probably overfitting the data set. Still, if we take into account our observations on the results of our previous research (Lankveld et al. 2011a), in which about 40 test subjects participated, and a repetition of that experiment with another 40 test subjects (Lankveld 2012), we see at least three general observations cropping up con- sistently: (1) High openness generally corresponds to fast exploration; (2) High agreeableness corresponds to friendly conversations; and (3) High neuroticism corresponds to tak- ing long to finish the game. Both the first and the third of these observations clearly held for the present experiment; the second was less obvious, but the game that we used here did not offer many opportunities to pick friendly conver-

183

sation options. As such, the present experiment should be considered a validation and generalization of our previous research.

The correlations that we discovered seldom reach large effect sizes (0.5 or higher) but they are significant (Balemans 2011; Lankveld 2012). One explanation is that Fallout 3 is not designed primarily as a personality test. The player’s behavior in the game is influenced not only by the situations that he encounters in the game and his personality, but also by many outside factors. Because of this it is reasonable to expect low correlations when looking at any single factor. We wanted to see whether personality effects can be found in a commercial game and we have succeeded in that. Previous reseach indicated that trait prediction can be improved by putting situations in the game that are optimally suited to each trait’s expression (Lankveld et al. 2011b).

Based on the high number of significant correlations, we tentatively conclude that player personality is expressed by game behavior, and it is possible to build a model of some of the player’s personality traits (in particular Openness and Neuroticism) automatically based on observations on the player’s game behavior. In future research (which we are currently performing) we will examine which behaviors that are indicative for personality traits generalize across games.

Conclusion In our research we tested 36 participants on 165 game be- havior variables observed in the game Fallout 3 and on the NEO-FFI personality questionnaire. From our results we may conclude that personality effects on game behavior ex- ist for all five traits of the Five Factor Model. These effects were particularly strong for the traits Openness and Neur- oticism. Our conclusions validate and generalize the conclu- sion that we reached when performing similar research on a game that we designed ourselves for measuring personality traits (Lankveld et al. 2011a). We may conclude that game observations can provide sufficient insight into personality to construct a valid personality profile of a player.

To explore the concept of measuring personality through observations of game behavior further, a new, large data set is needed with observations of players of several differ- ent games, accompanied by personality questionnaires and other written tests completed by these players, demograph- ical data, and biometrics measurements. Setting up such a data set is our first aim for future research. This will be a time-intensive process, but the present results indicate that such an effort has strong potential to lead to valuable results.

References Back, M., and Egloff, B. 2009. Yes we can! A plea for direct behavioural observation in personality research. European Journal of Personality 23(March):403–405. Balemans, I. 2011. Psychological Player Profiling: Re- search on measuring personality traits through observation of behavior in a commercial game. BA thesis, Tilburg Uni- versity, Tilburg, NL. Bateman, C., and Boon, R. 2005. 21st Century Game Design. Charles River Media.

Canossa, A. 2009. Play-Persona: Modeling Player Beha- viour in Computer Games. Danmarks Design Skole. Costa, P., and McCrae, R. 1995. Domains and facets: hierarchical personality assessment using the revised NEO personality inventory. Journal of personality assessment 64(1):21–50. Digman, J. 1990. Personality Structure: Emergence of the Five-Factor Model. Annual Review of Psychology 41(1):417–440. Hoekstra, H.; Ormel, J.; and De Fruyt, F. 2007. NEO-PI-R / NEO-FFI Handleiding. Hogrefe Uitgevers B.V., Amster- dam, The Netherlands. John, O., and Srivastava, S. 1999. The Big-Five trait tax- onomy: History, measurement, and theoretical perspectives. Oxford University Press. chapter 4, 102–138. Kolar, D.; Funder, D.; and Colvin, C. 1996. Comparing the Accuracy of Personality Judgments by the Self and Know- ledgeable Others. Journal of personality 64(2):311–337. Lankveld, G.; Spronck, P.; van den Herik, H.; and Arntz, A. 2011a. Games as personality profiling tools. In Preuss, M., ed., 2011 IEEE Conference on Computational Intelligence in Games, 197–202. Lankveld, G. v.; Schreurs, S.; Spronck, P.; and van den Herik, H. 2011b. Extraversion in Games. CG’10 Proceed- ings of the 7th international conference on Computers and games 1:263–275. Lankveld, G. v. 2012. Quantifying Individual Player Dif- ferences. PhD thesis, Tilburg University, Tilburg, NL (forth- coming). McCrae, R., and Costa, P. 1997. Personality trait structure as a human universal. The American psychologist 52(5):509– 16. Mount, M.; Barrick, M.; and Strauss, J. 1994. Validity of observer ratings of the big five personality factors. Journal of Applied Psychology 79(2):272–280. Schreurs, S. 2009. Psychological Profiling of Game Players. BA Thesis, Tilburg University, Tilburg, The Netherlands. Ten Berge, M., and De Raad, B. 2002. The structure of situations from a personality perspective. European Journal of Personality 16(2):81–102. Thue, D.; Bulitko, V.; Spetch, M.; and Wasylishen, E. 2007. Learning player preferences to inform delayed authoring. In AAAI Fall Symposium on Intelligent Narrative Technologies. AAAI Press, Arlington. Thurstone, L. 1934. The vectors of mind. Psychological Review 41(1):1. White, J.; Hendrick, S.; and Hendrick, C. 2004. Big five personality variables and relationship constructs. Personal- ity and Individual Differences 37(7):1519–1530. Wiggins, J. 1996. The five-factor model of personality: The- oretical perspectives. The Guilford Press. Yee, N. 2006. Motivations for play in online games. Cyber- Psychology & Behavior 9(6):772–775.

184