Urban planing history: event reflection

This article was downloaded by: [Arizona State University] On: 13 February 2015, At: 10:44 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Click for updates

Journal of the American Planning Association Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/rjpa20

Planning and Social Media: A Case Study of Public Transit and Stigma on Twitter Lisa Schweitzera a University of Southern California Published online: 16 Dec 2014.

To cite this article: Lisa Schweitzer (2014) Planning and Social Media: A Case Study of Public Transit and Stigma on Twitter, Journal of the American Planning Association, 80:3, 218-238, DOI: 10.1080/01944363.2014.980439

To link to this article: http://dx.doi.org/10.1080/01944363.2014.980439

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Problem, research strategy, and fi nd- ings: How media portray public transit services can affect the way voters and stakeholders think about future transit investments. In this study, I examine social media content about public transit from a large sample of Twitter comments, fi nding that they refl ect more negative sentiments about public transit than do the comments about most other public services, and include more negative material about transit patrons. However, transit agencies may be able to infl uence the tone of those com- ments through the way they engage with social media. Transit agencies that respond directly to questions, concerns, and com- ments of other social media users, as opposed to merely “blasting” announce- ments, have more positive statements about all aspects of services and fewer slurs directed at patrons, independent of actual service quality. The interaction does not have to be customer oriented. Agencies using Twitter to chat with users about their experiences or new service also have statisti- cally signifi cantly more positive sentiments expressed about them on social media. This study’s limitations are that it covers only one social media outlet, does not cover all transit agencies, and cannot fully control for differences in transit agency service. Takeaway for practice: Planners committed to a stronger role for public transit in developing sustainable and equita- ble cities have a stake in the social media strategy of public transit agencies; moreover, they should not let racial and sexist slurs about patrons dominate feeds. Planners should encourage interactive social media strategies. Even agencies that only tweet

Planning and Social Media

A Case Study of Public Transit and Stigma on Twitter

Lisa Schweitzer

H ow individuals, newspapers, and television portray public services can affect the way voters and stakeholders think about planning and public services. Attitudes and beliefs can infl uence politicians mak-

ing budgetary and other decisions such as public transit investment. Word of mouth can also affect whether consumers choose a particular service (Aday, 2010; Ajzen & Fishbein, 2005). With social media, individuals as well as traditional news media, businesses, and other institutions create the content that people view. I examine one social media outlet, Twitter, to see what social media users say about public transit and whether transit planners and agencies can infl uence the content on social media.

This study asks and answers the following questions:

• Do social media users describe transit planning, management, and ser- vices in a positive or negative manner?

• Do differences in social media interactions infl uence the tone of the discussion surrounding agency services, planning, and public manage- ment on social media?

• If so, should planners get more actively involved in social media to foster positive messages about public transit services and maintain civil dialog about patrons?

I fi rst discuss media effects and their potential infl uences on sentiments, attitudes, and choice behaviors. The third section of the study describes my use of a machine-learning algorithm; I show that social media content about transit, in a large sample of nearly 64,000 comments made on Twitter, refl ects more negative sentiment than comparable content about parks and airlines.

interactively a few times a day seem to have more civil discussions surrounding their agencies and announcements on Twitter than agencies that use their feed only to blast service announcements. Keywords: public transit, social media, racism, sexism, planning ethics About the author: Lisa Schweitzer ([email protected]) is an associate

218

professor at the Sol Price School of Public Policy, University of Southern California. She blogs at www.lisaschweitzer.com and is on Twitter as @drschweitzer.

Journal of the American Planning Association,

Vol. 80, No. 3, Summer 2014

DOI 10.1080/01944363.2014.980439

© American Planning Association, Chicago, IL.

RJPA_A_980439.indd 218RJPA_A_980439.indd 218 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 219

Sentiment surrounding public transit appears similar to that about police departments and social welfare programs. I also fi nd that the negative commentary on Twitter in- cludes more negative material about transit patrons than that of other public or private services. Comments about race, class, gender, and other markers of discrimination occur more frequently in the social media content on transit than they do for any of the other services examined, including social welfare programs. In fact, if racial, ageist, and other slurs against patrons are removed from the tweets, airlines and public transit agencies have statistically indistinguishable sentiment scores on social media. Transit advocates have long maintained that transit is viewed with a social stigma, and the evidence from social media sug- gests that advocates may be right.

The fourth section explores whether the ways in which transit agencies communicate via social media might infl u- ence the tone or sentiment of comments made by other social media users. By hand coding 5,000 randomly selected tweets, I test whether transit agencies that engage in more interactive dialog with other Twitter users have more posi- tive statements than those agencies that simply “blast” out service and related announcements. In this study’s penulti- mate section, I show that the answer is, tentatively, yes: Agencies with more interaction between transit agency representatives and other social media users have statistically signifi cantly more positive statements about all aspects of service—and fewer racist and sexist comments—than those agencies that simply blast information. Even using Twitter just for customer service is correlated with more positive feelings about the agency on social media.

Overall, I fi nd that how transit agencies use social media seems to infl uence user response and affects demo- cratic engagement with public transit support and invest- ment. If planners seek to support strong public transit systems as a key element in building equitable and sustain- able communities, they should encourage positive public sentiment about the service, in part by encouraging public transit agencies to use interactive social media approaches. This should not require extensive resources. Even agencies who only tweet a few times a day, but who use that pres- ence to chat with people rather than make announcements, seem to have more civil discussions surrounding their agen- cies and announcements on Twitter.

Media Effects, Attitudes, and Beliefs About People, Places, and Services

The media effects research is immense because it has accrued over six decades in communications, journalism,

and psychology. Media in Americans’ daily lives are perva- sive. According to Potter (2012), there are 65.5 million hours of original programming produced for radio each year; television produces 48 million hours. There were 13.6 billion websites in Potter’s census. Given that volume and variety of content, exposure—the time that individuals spend engaging with media in some way—is vast. Philips (2010) fi nds that, on average, Americans spend 11 hours exposed to various forms of media every day, including the Internet.

Research on media effects examines what infl uence all these different images, words, and ideas can have on indi- viduals (Ball-Rokeach & DeFleur, 1976; Drew & Weaver, 1990). Infl uence is defi ned as the ability that media con- tent has to alter people’s behaviors, emotions, attitudes, and beliefs in such a way that they become part of the way a person processes information and makes judgments (Scheufele, 1999). According to Potter (2012), there are more than 10,000 published studies on media effects that show mixed results, as with most large research literatures. Results range from substantial to negligible effects depend- ing on the context, media format, message, and audience. Nonetheless, it is fair to say that there is a consensus in the communications research that under the right conditions, media does infl uence what people think about and, in turn, how they evaluate the things media brings to their attention (Gunther, 1991).

Online services and social media represent a compara- tively new arena for public agencies seeking to infl uence how people think about public services such as transit. Many public transit providers have already moved into social media for various management and communica- tions activities, such as service announcements and cus- tomer service (Bregman, 2012; Collins, Hasan, & Ukku- suri, 2012; Mai & Hranac, 2013; Moss & Kaufman, 2013). Social media, however, present a challenge for communicating specifi c messages and values. Unlike advertising, where transit agencies can craft their own images and messages to disseminate, social media content comes from many users. Understanding that content can help us in turn to understand what people say, both good and bad, about urban issues and public services such as transit. We can then use that understanding to employ social media more effectively in planning.

Table 1 displays Twitter content results from a simple name search on SEPTA, the Southeastern Pennsylvania Transportation Authority serving the Philadelphia region. I selected this particular group of tweets because it displays the issues in play here. Pronouncements from users about how SEPTA generally runs late are interspersed with SEP- TA’s announcements about particular service problems. The

RJPA_A_980439.indd 219RJPA_A_980439.indd 219 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

220 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

content returned in this search refl ects three dimensions of media content that can infl uence attitudes and beliefs: the message a) highlights problems and is thus framed nega- tively; b) is consistent; and c) comes from multiple sources that echo the same message: Service is late. Taken together, that consistent message about late service may not be good marketing for the service, the agency’s public management, or their plans. I do not use the agency’s own content in the analyses of what users say (described below), but I show it here to demonstrate how agencies contribute to the negative content with their own messaging.

What messages emerge from all this disparate con- tent? Most prior research on online content suggests that negative comments dominate just about all online mate- rial (Hu & Liu, 2004). Complaining online, from overly entitled Yelp customers to misogynist 4chan users, is to some degree expected in online commentary. Complain- ing about transit service is hardly novel or unexpected among those who use the service every day. People can be unreasonable in what they say about all types of service, transit included (Weinstein, 2000). They can also be right in that service can and should be better. Since complain- ing online appears to be the norm, just fi nding com- plaints demonstrates little about the overall content. The question is whether, relative to other topics, transit or other public services contain disproportionately more negative content, or particularly vivid negative images and content, that might lead us to believe complaining about transit is somehow worse or different than it is for other public services.

Methods: Machine Learning and Text Mining

I use data I collected from the social media site Twitter (https://twitter.com/) to analyze whether comments about transit service are more negative than comments made other public services. Twitter is a microblogging social media service where individuals post messages of 140 characters or less, images, and links to other content. Twitter allows users to “follow” other users and receive their content automatically, though anyone can view Twitter content just by searching the site. Users can mes- sage each other freely.

Throughout, it is important to remember that Twitter users are not representative of wider transit patron groups, voters, or the U.S. public. Twitter users are younger, more affl uent, and more educated than the U.S. population as a whole (Brenner & Smith, 2013). Nonetheless, according to a 2013 survey, one in every 10 American adults reports getting their news from scanning Twitter (Mitchell & Page, 2013).

In addition, Twitter users, as an audience, are only one of many ways Twitter comments can have infl uence. Twitter comments, unlike those on Facebook, are public and accumulate into a searchable short-term text archive that even those without Twitter accounts can search and access. As a result, Twitter content may exert dispropor- tionate infl uence on other, traditional media because Twitter allows journalists, who are important media content creators, to draw on Twitter content for their stories.1 Thus, Twitter yields a sample of readily available, readily accessed content that can cross over into other media.

I use two related methods called machine learning and text mining to evaluate the content of the tweets I exam- ined. The text-mining method uses an algorithm to scan the Twitter text to match words found in the Twitter comments against an existing lexicon of positive and negative opinion words. For this analysis, I take a basic counting approach starting with an established lexicon of positive and negative English and Spanish words developed by computer scientists (Hu & Liu, 2004). Their dictionary has been used extensively in prior research, and it is avail- able via free distribution online at http://www.cs.uic. edu/~liub/FBS/sentiment-analysis.html. The algorithm scoring method is described via formulae in the Technical Appendix. The programming code needed to conduct the analysis in the open source programming language R is available from the author.

Machine learning describes how a computer selects information based on an algorithm that can be

Table 1. Public search results for SEPTA.

SEPTA @SEPTA Chestnut Hill West: Train #826 going to Fox Chase is operating 12 minutes late. Last at Chestnut Hill West.

Twitter User #1 When I say trust no one I really just mean don’t trust septa to get you to work on time.

Bob Kelly @ bobkellytraffi c

Scattered delays on SEPTA’s Regional rails.

SEPTA @SEPTA Newark: Train #4213 going to Newark is operating 16 minutes late. Last at Temple U.

SEPTA @SEPTA West Trenton: Train #319 going to Elwyn Station is operating 11 minutes late. Last at Fern Rock TC.

Twitter User #2 it don’t even be my fault why I’m late. where tf the trolley at yo, I really hate septa.

Twitter User #3 A SEPTA bus before 6 am is the weirdest. Feel like I’m going on a fi eld trip with a group of senior citizens.

Note: Agency tweets are included here for display, but they are not included in the analyses done throughout this manuscript.

RJPA_A_980439.indd 220RJPA_A_980439.indd 220 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 221

programmed to learn, either organically from the data or from a human that provides feedback to the algorithm iteratively. For example, the spam fi lter for email works by machine learning. Here, I combine machine learning to improve the text-mining methods. I wrote a second algo- rithm that examined frequently occurring text strings to do two things: 1) identify tweeters who may be outliers in terms of the amount they tweet; and 2) add words to the lexicon that might be unique to the conversations about transit. I used an interactive machine-learning method so when the algorithm detected an unusually frequent text string, it sought my input as to whether to mark that string or not. Locating frequent tweeters can help pinpoint satire or parody accounts that exist to mock or denigrate transit agencies: Those do exist. Other frequent text strings are words likely relevant to transit service but not included in the original lexicon.

The resulting additions included four positive words: congrats, congratulations, thanks, and upgrade. The interac- tive machine again yielded far more negative words than positive words, and these, too, went into to Hu and Liu’s (2004) original listing in the sentiment scoring: brokedown, wtf, late, wait, waiting, delay, delayed, offl ine, scam, closed, epicfail, weirdo, pervy, crowd, crowded, jammed, skeevy, breakdown, ghetto, transitfail, and unsuck. The modifi ed lexicon then replaced the original lexicon used to score opinions found in the Twitter content.

I used the text-mining algorithm to score the text strings for opinions based on the lexicon derived from Hu and Liu (2004). The algorithm gives a zero score to content that has no positive or negative words, or that offsets positive and negative words within the text (Pang & Lee, 2008). Otherwise, the algorithm marks words as positive or negative and then sums those scores for the total content. “I love the metro,” for example, would be coded 1. Conversely, “I hate the metro” would be coded –1. “The metro is late” would be scored –1, whereas “Great ride, Metro” would be scored 1. The algorithm would score something like “Super awesome great ride” a 3.

Machine coding is imperfect, so slang, misspellings, and repetitions pose a challenge. It is not clear that “I lurve Tri-Met!” is more or less positive than “I love Tri-Met” or “I love love love Tri-Met!” Most opinion-mining methods, like this one, just count positive and negative words; there- fore, the last tweet would be counted as more positive than the fi rst tweet. The system also does not handle sarcasm particularly well. However, machine coding draws on such a large body of text information that these stylistic quirks and errors become less important than what emerges from the large volume of data as empirical regularities.

Sampling

I used a two-part sampling strategy. The fi rst sample selects a set of controls from two groups likely to have extremely positive comments (“celebrities”) and extremely negative comments (“villains”). Because text mining in- volves thousands of text strings, we need the celebrity and villain controls to see whether the opinion-scoring algo- rithm properly recognizes words and then scores them the right way for positive and negative content. If the algo- rithm performs properly, opinion scores for celebrities should all group together and be strongly positive, while the opposite should be true of the villains.

The second sampling strategy compares transit against other public and private services as test groups. Table 2 lists all the agencies and terms used in the test groups. The transit agencies in the test group were randomly sampled from the American Public Transportation Association’s list of top 30 passenger-serving transit companies.2 Airlines serve as a control group because, though airlines are private companies, they have multiple service parallels with public transit. Air travel is, like transit, a ticketed service where individuals might comment on service timing, staff con- duct, ticketing, facilities, and other passengers. In addition, I sampled content about urban public parks to capture discussion about public space as an overlap with transit as a public venue. I also randomly selected police departments as a control, acting as another local government institution. My fi nal control group is social welfare programs. The latter are likely to carry a stigma within U.S. public and political dialogs. I sampled content in winter, summer, fall, and spring months from 2010 until 2014, as the com- plaints related to parks and transit are seasonal. The fi nal sample contains 63,321 comments. More specifi c informa- tion on the machine coding used in this study appears in the Technical Appendix.

Figure 1 shows the average sentiment score found for all the content sampled from Twitter for each agency or person in the sample. To make the fi gure clearer, I display only fi ve indicators for the controls: the top three and bottom two of each category. The dashed line in the center of the fi gure shows the 0 score; the thicker horizontal line indicates where average scores switch from positive to negative. Zero scores indicate no real sentiment. Positive scores indicate positive sentiment; negative scores refl ect entities where the content runs, on average, negative. All tweets from all offi cial agency Twitter accounts were removed from these data. Figure 1 suggests that the machine-coding algorithm worked properly because it produces scores that matched my expectations for the likely extremes. That is, in general, celebrities were at the positive end of the distribution (William Shatner and the

RJPA_A_980439.indd 221RJPA_A_980439.indd 221 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

222 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Kardashians), whereas villains mark the other end of the scoring distribution. A random sample of tweets checked by hand also verifi ed that the algorithm scored the material properly, so that the differences seen here refl ect differences in the data, not coding problems.

The second set of controls involved airlines and other public services (the police, social welfare programs, and parks); they group together in ways that suggest that the machine learning discovered differences in positive and negative statements about these services. In general, people tweet very positive things about parks. The picture is less clear about airlines, however; Alaska Airlines and South- west have positive sentiment scores, near those of celebri- ties and parks. US Airways and United, however, have negative mean sentiment scores. The airline rankings mirror those found in more in-depth customer satisfaction surveys reported by JD Powers and Associates, with Alaska Airlines leading among traditional airlines and Southwest scoring well among small regional services.3

I found signifi cant differences in the opinions ex- pressed about these control groups in the Twitter commen- tary. The details of the analysis are in Table A-1 in the Technical Appendix. The results show that the sentiment expressed about public transit on Twitter is more negative

than for parks and airlines and indistinguishable from police and social welfare programs.

Figure 2 breaks comments into categories, the fi rst related to aspects of service, the second to the personal aspects of the people involved in the service. Twitter com- menters complain about both service and fellow passen- gers—a lot. Tweets about social welfare programs and public transit contain comments about race, class, and gender that do not appear as frequently in tweets about parks or airlines, where comments about age predominate in complaints about fellow passengers. Gendered racial slurs about African-American women are disproportion- ately frequent in the transit and social welfare content.

To examine the potential sources of negative commentary, a second machine-coding exercise isolated all the sentiment, both positive and negative, about the personal appearance of the patrons within the data set. I found that the differences in sentiment about transit and airlines become statistically indistinguishable once I remove slurs from the data. (See Table A-1 in the Technical Appendix.) In short, negative and racist comments about transit patrons are a major component of all negative comments about transit; otherwise, commenters appear to be saying equally happy and unhappy things about public transit service and airline service.

Table 2. Control groups and test groups for machine scoring.

Control groups

Test groups

Public parks Transit

Social programs/

stigma Airlines PoliceVillains Celebrities

Transportation Security Administration (TSA)

Serena Williams (athlete)

Grant Park (Chicago) MBTA (Boston)

Food stamps American Atlanta

Osama Bin Laden (leader of terrorist strike against the U.S.)

William Shatner (actor)

Bryant Park (New York)

NY MTA (New York)

AFDC Alaska Air Baltimore

Internal Revenue Service (federal auditing service)

Kardashian (reality TV stars)

People’s Park (Berkeley)

CTA (Chicago)

Social Security British Airways

Cincinnati

Westboro Baptist Church (organization that protests at high-profi le funerals)

Snoop Dogg (musician)

Prospect Park (Brooklyn, NYC)

Translink Medicare Delta Chicago

Michael Dunn (accused killer) Nathan Fillon (actor)

Audubon Park (New Orleans)

TTC (Toronto)

“minimum wage” JetBlue Detroit

Logan Circle (Philadelphia)

SEPTA Medicaid United Los Angeles

Forest Park (St. Louis) DC Metro “welfare queen” US Airways Miami

Golden Gate Park (San Francisco)

BART (San Francisco)

Affordable Care Act

Virgin America

Oakland

Boston Common (Boston)

TriMet (Portland)

Obamacare Southwest Philadelphia

Patterson Park (Baltimore)

LA MTA Sacramento

Note: AFDC = Aid to Families with Dependent Children.

RJPA_A_980439.indd 222RJPA_A_980439.indd 222 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 223

Figure 1. Opinion score rankings that result from machine scoring.

RJPA_A_980439.indd 223RJPA_A_980439.indd 223 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

224 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Figure 2. General topics covered by the Twitter comments.

RJPA_A_980439.indd 224RJPA_A_980439.indd 224 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 225

sign that reads, “For more effi cient service, please get off at the next stop, where a team of heavily drugged sloths will drag you to your destination.” The machine code results in a +2 score because of the words “eloquent” and “inspiring” in the actual tweet, but the picture sends a different mes- sage. Hand coding scored this tweet a –1 for service tim- ing. This error occurred more often for transit than for any other control, so that the match between hand and ma- chine coding was 82%, with most of the mistakes occur- ring with images. The resulting distributions placed hand- coding scores lower than the machine coding, but do not affect the results of statistical analysis. Between this check and the celebrities and villains controls, I can conclude that the algorithm I am using is not driving the results. Instead, there are statistically signifi cant differences among transit agencies and between controls.

Communication Style and Potential Infl uence on Sentiment

As Table 4 illustrates, transit agencies vary in how they use their Twitter accounts. The fi rst or “interactive” style entails chatting in two-way communication with individ- ual commenters. By contrast, blast communications tend to be one-way communications from the agency outward, with little patron interaction. It includes many service announcements and some social marketing and other agency self-promotion. Some agencies, such as SEPTA and

Table 3. Basic agency performance data.

Score rank Name

On-time performancea

Unlinked Trips (millions)b

1 Translink 90% to 95% 237

2 TTC 55% to 90% 471

3 TriMet 76% to 98% 99

4 LA Metro 78% 476

5 BART 88% to 94% 127

6 WMTA 79% to 91% 413

7 NY MTA 77% to 91% 3,467

8 SEPTA 87% 358

9 MBTA 98% to 100% 395

10 CTA No report 529

Notes: a. Some agencies (e.g., LA Metro) report only systemwide performance, while others disaggregate by line. For the latter, a range is reported (Bay Area Rapid Transit, 2014; Los Angeles Metropolitan Transportation Authority, 2014; Massachusetts Bay Transportation Authority, 2014; New York Metropolitan Transportation Authority, 2014; South Coast British Columbia Transportation Authority, 2014; SEPTA, 2014; Toronto Transit Commission, 2014a, 2014b; Tri-Met, 2014; Washington Metropolitan Area Transit Authority, 2014). b. Data from the American companies come from the National Transit Database (2013), data compiled by the author. Translink data come from South Coast British Columbia Transportation Authority (2013).

Does a difference in service quality account for the differences in tweets about individual transit agencies? Table 3 displays the most recent data about transit agency service along with self-reported on-time performance data. Neither of these two dimensions of service provision tracks closely with the ranks that emerge from the Twitter com- ments. It is not that poor service providers are more criti- cized on social media while the “angel” companies provid- ing on-time service get the appreciation they deserve. In fact, the more service an agency provides, the more com- plaining there is. My approach measures the content of the conversation about the agencies, not their service quality, per se. Thus, the differences in opinion expressed about transit companies cannot be explained by differences in service quality.

I hand coded the content using a random sample (n = 1,000) of the machine-coded comments to double check how well the machine-coded approach performed against human coding. This also allowed me to view more detail about the source and content of the tweets. In general, hand and machine codes matched well except for places where the machine coding would be expected to fail, such as with sarcasm and images. One tweet in particular illus- trates multiple problems for machine coding: a picture of a

Table 4. Different communication styles.

“Dialog”-style communication

SEPTA_SOCIAL @SEPTA_SOCIAL Oct 12

Good Morning! I’m Eric of SEPTA’s Social Media team ready to assist you. Bleed Green, Cheer Green! Go Eagles! Tweet us if you need us!

Twitter User Oct 12 @SEPTA_SOCIAL Hi Eric! Will septa run on a holiday schedule tomorrow?

SEPTA_SOCIAL @SEPTA_SOCIAL Oct 12

We’re operating on a weekend schedule tomorrow.

Twitter User Oct 12 @SEPTA_SOCIAL thank you!

SEPTA_SOCIAL @SEPTA_SOCIAL Oct 12

You’re welcome! Have a pleasant Sunday!

“Blast”-style communication

cta @cta 9h Remember: The O’Hare station reopened yesterday afternoon, following repairs. ow.ly/vdd07

cta @cta Mar 30 The O’Hare station is open for service. Here are some pics of repair work and the reopening:

RJPA_A_980439.indd 225RJPA_A_980439.indd 225 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

226 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

the Toronto Transportation Commission (TTC), break their customer service and service notices into separate feeds.4 Most transit agencies have thousands of followers who see what the agencies post, but only a few of them (Translink, Bay Area Rapid Transit [BART], and TriMet) subscribe or follow a large number of feeds themselves. It is not required that agencies follow other users to interact with them, but following is one way to signal to other users that one wants to see their content. Translink is by far the most active Twitter user among the transit agencies that I studied. They tweet often, on average about 90 times per day, while most other agencies tweet between 10 and 30 times a day. Translink interacts a great deal in conversation with other Twitter users, and they also follow others more than other transit agencies. TriMet also follows a compara- tively high number of other Twitter users compared with other transit agencies.

One agency’s approach nicely illustrates what interacting with patrons might do to alter the content related to an individual transit agency. I fi rst began to collect Twitter comments in 2010; I continued to do so through 2014. In the fi rst years of the study, the opinions expressed about SEPTA always ranked at or near the bottom with the Massachusetts Bay Transportation Authority (MBTA) and Chicago Transit Authority (CTA). In December of 2011, SEPTA introduced its customer service dialog feed (@SEPTA_SOCIAL) to run parallel with its blast feed on service announcements. After one year using a two-way communication style, SEPTA’s opinion score had gone from –0.3 (not far from the “villains” control) to ranging from 0 to –0.09. Although still negative, that is a 70% improve- ment in a relatively short amount of time. We can see that this change is more signifi cant when we examine the effect on the second source of negativity: slurs directed at other passengers. Prior to the customer service feed, the race, class,

and gender categories held nearly 7% of SEPTA’s total feed in data collected from winter 2010 to 2011. From fall 2012 through spring 2014, that percentage had shrunk to 2.5%.

To check for differences in communication style, I randomly sampled 60 days of the content of all Twitter feeds that came directly from the agency itself over the course of two years. I then identifi ed which agency Twitter comments were “conversation” moments, in which an agency representative tweets back and forth with another commenter. I then developed a dialog score (D-score in Table 5) that calculated the ratio of conversation moments to the agency’s total content broadcast via Twitter to every- one. Table 5 displays the D-score for each of the 10 agen- cies sampled. Based on that score, the agencies were grouped in levels of interaction with other Twitter users: high (D < 60%), medium (30% to 60%), and low (less than 30%). TriMet tweets comparatively little, but when they do, it is often to interact with other Twitter users. It is important to remember that the D-score is a measure of relative effort and does not measure overall effort. It simply refl ects what percentage of the agency’s total communica- tion on Twitter occurs in dialog with other Tweeters as opposed to being blasted out.

Hand Coding Twitter Content by Topic and Sentiment

To test whether there were statistically signifi cant differences in the communication style of various agencies, I drew a sample of 5,000 tweets, 500 for each agency, from the entire Twitter data set, excluding tweets from the agencies’ own feeds. I stratifi ed the sample by Twitter user profi le: government, elected offi cials, business accounts, nongovernmental organizations (NGOs), patrons, transit

Table 5. Online interaction differences among transit agencies.

Machine score ranking Agency D-score

Interaction category

Followers (1000s) Following

1 Translink 0.86 High 50.3 12,200

2 TTC 0.77 High 12.7 492

3 TriMet 0.40 Medium 15.0 8,350

4 LA Metro 0.62 High 26.1 329

5 BART 0.13 Low 59.5 2,246

6 WMTA 0.27 Low 65.9 228

7 NY MTA 0.00 Low 96.4 243

8 SEPTA 0.53 Medium 7.01 1,661

9 MBTA 0.41 Medium 15.9 9,030

10 CTA 0.22 Low 43.0 62

RJPA_A_980439.indd 226RJPA_A_980439.indd 226 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 227

operators, planners, enthusiasts, and watchdogs and satire accounts. Table A-2 in the Technical Appendix provides more detail about the sampling stratifi cation method and how I coded by user type.

My graduate student coders and I met initially to select a coding rubric together. The two students and I fi rst coded the same 200 comments separately. The team then applied multiple codes to the Twitter text to come to a higher leveling of coding consistency. Several iterations were required for the team to reach an acceptable Krippen- dorf ’s alpha of .86.5 After this standard was met, the team coded all the data and, again, backcoded a random sample to test consistency. This check again yielded a Krippen- dorf ’s alpha of 0.84, which is acceptable.

Table 6 shows a selection of coded Twitter comments to illustrate the coding method, also outlined in greater detail in the Technical Appendix. The research team coded Twitter comments by the type of service complaints: tim- ing, driver content, staff conduct, facilities, security, and

climate control. We also coded complaints about the behavior of passengers as well as slurs about passengers. Slurs include comments about age; size; race; class; gender; disability; size; and lesbian, gay, bisexual, or transgender (LGBT) status. Finally, the team marked content regarding planning and public management announcements.

Table 7 shows the average for each variable by high, medium, and low interaction groups previously described. Opinions about service, patrons, planning, and manage- ment, while still negative, are consistently higher for agen- cies in the high-interaction group, those agencies that engage more directly with individual tweeters.

Table 8 summarizes fi ndings of statistical tests of opinion by topic and by interaction level. These tests illustrate two important things: Transit companies that respond to other social media users have statistically more favorable opinions expressed about the transit agency for just about every measure I considered. (The statistical tests are displayed in full in Tables A-3 and A-4 in the Technical

Table 6. Examples of hand coding.

Code (score)a Commentary

Timing (–2) @wmata Slowly sucking our souls and our money one messed up day at a time. 45 minute wait for Orange-line train today!

Facilities (–2) Just watched a man on crutches hobble down a broken escalator at L’Enfant. @wmata is despicable. #unsuckdcmetro

Driver conduct (–1) ICYMI: Operators toss pee-fi lled bottles on tracks. #wmata #vienna (via @Yarsh29) pic.twitter.com/YbfQ9uSY4x

Staff conduct (–2) The rudeness of the station manager at the Dupont Circle station is unbelievable. Why must you be so bitchy? @wmata

Facilities, staff conduct (–2)

Only 2 working fare gates for entry at Vienna. Station mgr watching, with enjoyment, as long lines develop & riders get frustrated. #wmata

Climate, disability, age (–4)

Ride on the #15 bus: it was late, crowded, no a/c and the old lady in the wheel chair who mumbles incoherently was on board. Joy. #Trimet

Security (–2) Two people stabbed on Red Line this morning. This is the third stabbing on #metrolosangeles rail in recent months.

Planning (1) MT @bccycle: New bike storage room for 50+ bicycles planned at Commercial-Broadway #transit Station!

Public management (–2) #Translink a disgrce 2 txpayrs & democrcy. Unelectd/Unaccountable - power 2 tax. “Card system delayed #vanpoli #BCpol

Age (–1) “Old people on septa” should have countless videos on youtube...

Behavior (–2) People are so stupid on septa. Like if you see there’s more room somewhere else why wouldn’t you say excuse me to get there? dumb {redacted} ppl

Behavior, service (–2)

I swear some people have their Earbuds in facing the wrong direction! I don’t want to hear your crap music on @SFBART ! Ride sux as it is!!

Class (–1) Whomever coined the term “the great unwashed” must’ve been riding the CTA. Chicago buses keep Typhoid, Tuberculosis and Whooping Cough alive

Class, race, behavior (–3) Ugh these ghetto dirty {redacted} kids all loud Looking The {redacted} Terrible I hate metro

Class, race (–1) Theres some GHETTO {redacted} people on the bus/metro. They really gdaf.

Gender, class, race (–2) Loud,Ghetto Girls On The Metro <<<<<< Like Stfu’. It gets under my skin.

Gender, class, race (–2) This little girl is too cute, but I can hear her hoodrat mama’s entire conversation, so I’m scared 4 her future. #septa

Gender, class, race, size (–4)

Metro home of, fi tted caps, fake Jordan’s,no teeth having nikkas, fat {redacted—gender slur} in leggings, All the air forces, ghetto loud people, and uglies

LGBT (–2) I avoid anywhere gays may be in {redacted}: The transit, uptown, {redacted}, gay clubs, #CPCC, etc.

Note: a. Only codes with Krippendorff ’s alpha >0.80 included.

RJPA_A_980439.indd 227RJPA_A_980439.indd 227 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

228 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Appendix.) Transit agencies in the “high” interaction group receive more favorable opinions about service timing, driver conduct, facilities, and climate control than agencies that use their feeds solely for service and related announce-

ments. Agencies that interact individually with their pa- trons also receive more favorable opinions about patron behavior, and fewer slurs about class, race, gender, and size. There were, however, no statistically signifi cant differences between the groups in expressed opinions about LGBT, disability, or age. It may be that the smaller sample, coded by hand, was not large enough to capture these effects.

The results are consistent across multiple topics. Peo- ple who comment on social media about transit agencies use less negative language when the transit agency responds to individuals rather than using social media to blast an- nouncements. That effect is most consistently signifi cant for the agencies with the most two-way communication between agency representatives and other Twitter users. Tweets about planning and management information shared on Twitter are also more favorable for agencies that use social media as an opportunity to converse rather than announce.

My fi nal analysis examines the type of user who raises transit planning and public management topics on Twitter. Table 9 displays a summary of the statistically signifi cant pairwise comparisons for each group of transit user. (The full statistical tests appear in A-5 and A-6 in the Technical Appendix.) There are many differences in reactions to planning announcements on social media. Examples of planning content or announcements include tweets with updates on new facilities; calls for ideas, opinions, new projects, and plans; or invitations to participate in the planning activities of the agency.

Table 7. Differences in mean opinion score by agency interaction level.

Low Medium High

Score –0.5789 –0.4473 –0.2285

Timing –0.2115 –0.1834 –0.1604

Driver conduct –0.0391 –0.0298 –0.0140

Staff conduct –0.0158 –0.0119 –0.0047

Facilities –0.1192 –0.0885 –0.0028

Climate –0.0196 –0.0159 –0.0028

Security –0.0116 –0.0174 –0.0103

Age –0.0051 –0.0040 –0.0019

Behavior –0.1188 –0.0760 –0.0289

Class –0.0088 –0.0060 –0.0009

Disability –0.0019 –0.0030 0.0000

Race –0.0088 –0.0065 –0.0009

Gender –0.0098 –0.0040 –0.0019

LGBT –0.0014 0.0000 0.0000

Size –0.0075 –0.0010 0.0009

Planning –0.0815 –0.0442 –0.0065

Management –0.0214 –0.0124 0.0000

Table 8. Summary of signifi cant results by communication strategy.

Interaction level

High-low High-med Med-low

Average sentiment score Yes Yes Yes

Timing Yes — —

Driver conduct Yes — —

Staff conduct — — —

Facilities Yes Yes Yes

Climate Yes Yes —

Security — — —

Age — — —

Behavior Yes Yes Yes

Class Yes — —

Disability — — —

Race Yes — —

Gender Yes — Yes

LGBT — — —

Size Yes — Yes

Planning Yes Yes Yes

Management Yes Yes —

Table 9. Summary of statistical signifi cance by group.

Pairs compared Planning Management

Planning by interaction and group

High vs. low interactivity ✓

Operator-business ✓

Operator-enthusiast ✓

Operator-news media ✓

Operator-NGO ✓

Patron-enthusiast ✓ ✓ ✓

Patron-news media ✓ ✓ ✓

Patron-operator ✓

Planner-operator ✓

Planner-patron ✓

Watchdog-business ✓

Watchdog-enthusiast ✓

Watchdog-government ✓

Watchdog-news media ✓

RJPA_A_980439.indd 228RJPA_A_980439.indd 228 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 229

Members of traditional transit booster coalitions— planners, businesses, government agencies, enthusiasts, and new media—supply comments that vary signifi cantly from those of patrons, transit operators, and watchdog/satire accounts. Boosters, as might be expected, are consistently more positive; all other commenters are consistently more negative when they discuss planning topics on social media. These data provide evidence that there are transit booster coalitions who tweet positively about new facilities, new projects, and agency plans for construction. Planners, government agencies, news media, business owners, and transit enthusiasts all tend to tweet happy, enthusiastic things about public transit. The effect is much less appar- ent for public management topics such as budgeting and personnel changes.

Transit agencies that interact more on an individual basis with the public tend to get more participation in their feed from news media and businesses than those agencies that interact less often. Tweets from the news media and businesses are likely, on the whole, to be more favorable about transit. Thus, attracting allies such as business and news media accounts to transit can also boost the overall sentiment expressed on social media.

Bear in mind that these correlations are not proof of causation. It may be that transit agencies that have more positive sentiments have representatives who tweet more interactively because they are less likely to be insulted or criticized when they communicate. It is also possible, however, that the behavior of these agencies infl uences the online discussion for the better when they interact. I have examined only 10 transit agencies; the sheer volume of data available requires supercomputing capacity for analy- ses that really capture a comprehensive view of all the information for a more agencies. Nonetheless, the data cover nearly four years of comments, including 1,200 image fi les, made about transit by people on Twitter. These data capture the ongoing conversations about public transit on social media, not objective measures of service, but they suggest that transit agencies—and transit allies—might infl uence the tone and content of that conversation.

Conclusions for Planning

This research was designed to provide elevation and insights, if not comprehensive responses, to the following questions:

• Do social media users describe transit planning, management, and services in a positive or negative manner?

• Do differences in social media interactions infl uence the tone of the discussion surrounding agency ser- vices, planning, and public management on social media?

• If so, what role can and should planners play in shaping social media about public transit and its patrons?

In response to the fi rst question, I fi nd that the percentage of negative comments about transit service are roughly equivalent to those about police and social welfare pro- grams; moreover, many tweets contain racial, sexist, and other slurs about transit patrons, with African-American women targeted disproportionately. My analyses show that these slurs comprise such a large share of all negative comments about public transit that when I remove them, the differences in sentiment about transit and airlines become statistically indistinguishable. Differences in the level and quality of service provided by different transit agencies do not explain differences in the tweets about individual agencies. Finally, the machine-coding algorithm I am using is not driving the results; transit systems receive more negative comments than do most other public ser- vices. This is concerning because it may affect how a vari- ety of stakeholders view public transit both as a mobility option and as a public priority.

In response to the second question, I fi nd that people discuss the agency and its services more negatively when agencies blast information rather than interacting with individuals in a two-way, conversational tone. If media effects in public transit work the same way as they appear to work in other political topics—and it is a reasonable assumption that they do—then the extent and consistency of negative commentary found on Twitter may disadvan- tage transit as a mobility service relative to other choices available to travelers and within public opinion on the importance of supporting local transit agencies. My fi nd- ings mirror customer service research, which emphasizes that personalized attention infl uences the tone and civility of complaints; this effect may be especially true online (Grönroos, 2000).

The answer to the third question, the role that plan- ners can and should play in infl uencing social media, comes down to a normative evaluation of the obligation of professional planners to maintain civil dialog online about public transit patrons and public transit services. Planners, and the public transit agencies that they advocate for or represent, have choices in how they engage social media. They can try to go with “just the facts” in blasting out announcements, and in so doing, avoid dealing with people online. They can try to market themselves and their

RJPA_A_980439.indd 229RJPA_A_980439.indd 229 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

230 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

services; this marketing and self-promotion aspect to social media is obvious enough. They can also, as part of that effort, use social media to feed information to other gov- ernment agencies and news media. Just as important, planners can also use social media to “listen” and interact with their constituents and patrons. Or, planners can use social media for all three. However, simply blasting an- nouncements to advertise public transit service appears to be less effective in maintaining more positive content online than communication strategies that involve give- and-take with other users.

If planners wish to use social media to discuss agency plans, the future of transit, or what services offer, then these analyses suggest that planners or other agency repre- sentatives have to show some leadership in cultivating social media as a place for that to occur. Just as with more traditional methods of public outreach, day-to-day interac- tions online appear to foster more willingness to discuss planning, and to do so in a much more positive way. For those not familiar with Twitter, Table 10 shows a conversa- tion about planning between a patron and TTC Director of Communications Brad Ross in which the patron asks about the agency’s priorities and states an opinion about his own preferences. Both retweeting (Table 10) and con- versations occur more frequently and more civilly when, in general, transit agencies use interactive strategies on social media. It stands to reason: If agencies want people to talk to them about agency plans, agencies had better be pre- pared to interact with individuals to discuss their day-to- day concerns. Reciprocity is a part of dialog.

An additional aspect to leadership involves signaling civility. My analyses show that when transit agencies read and answer tweets they receive fewer and less severe slurs about transit patrons in general and about patrons from oppressed groups in particular, even if the agency only responds to a select number of tweets each day, the way TriMet and Los Angeles currently do. African-American women are, as a group, disproportionately slurred in the social media I sampled, even though they have been among

U.S. transit agencies’ most dependable patrons for decades (Hanson & Johnston, 1985; Williams, 2006). African- American women’s ideas and their lived experiences on transit strike me as too important for planners not to work to maintain civility on social media that counters the de- fault in social media, which appears to marginalize both transit as a service and African-American women as patrons.

An element of good news is that there seems to be little need to confront or otherwise try to censor racial, ethnic, or homophobic slurs. Transit systems that actively respond to individual patrons seem to have fewer slurs without directly addressing those slurs at all. Why this occurs is not clear. The reduction may occur because, by interacting with users over general topics, agency representatives remind people that they are communicating in a highly public forum in which anybody might read what they say. Or it may be that by both providing agency information and interacting about that information, agencies give people something to talk about other than other passengers and agency gaffes.

The analyses also show that transit agencies that en- gage actively with their patrons also get more positive attention from traditional news media outlets and other transit boosters, such as businesses. Here again, it is not clear from these data why this happens. Perhaps agencies interacting on social media have always had good relation- ships with their regional news outlets and local businesses, so the positive social media content refl ects ongoing rela- tionships. Also, news media and businesses that retweet or interact with transit agency social media staff may be trying to get their comments retweeted by the transit agency. Either way, transit agencies that do interact have more participation from traditional news media and local busi- ness, and their comments about the transit agencies are, on average, more positive than for those agencies that simply blast announcements.

These fi ndings suggest that planners and customer service representatives in transit agencies have opportuni- ties to leverage social media, but only when they are thoughtful. Very active agencies such as Translink tweet as much as 90 times a day; they do seem to have the highest payoff in terms of tone and content from other social media users as a result of that effort. Even agencies who only tweet a few times a day, but use that presence to chat with people rather than make announcements, seem to have more civil Twitter discussions about their agencies and announcements.

Simply blasting more tweets does not appear likely to be a way to improve how patrons comment about the service, agency, staff, and fellow patrons. Instead, agencies that blast service updates via Twitter may be doing

Table 10. Example of conversation about planning on Twitter.

Twitter User Sep 30 + doesn’t Eg line & Spdna exp just put even more pressure on Yellow line? Lots of opinions out there but TO needs the DRL @bradTTC

Brad Ross @bradTTC 5:30 pm 30 Sep 2014

To be clear, the relief line remains the TTC’s fi rst priority for new projects and funding.

Twitter User Sep 30 @bradTTC well hurry with that eh. I’m late for work! Haha. And thnx for always being available btw. #bradrossrules

Note: Brad Ross is communications director for TTC.

RJPA_A_980439.indd 230RJPA_A_980439.indd 230 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 231

themselves a disservice; this is particularly true among the larger transit agencies. They announce their problems to the world via social media rather than engaging with their patrons. Tweeting late service information is probably less effi cient than simply allowing smartphone applications to track on-time data; those applications can allow users to select the exact routes and services they want updates for, and when they want that information. This allows agencies to refrain from inundating users with information about all service disruptions the way most agencies do with Twitter. For the agency, the Twitter feed instead might be better used for building engagement and customer service, rather than blasting out simple service announcements that merely repeat what many patrons have already tweeted: A particular train is late.

My fi ndings present some challenges and opportunities for planners. On one hand, complaints about transit in high-profi le social media may tarnish both the mode and a specifi c agency’s reputation. Planners who advocate for transit services should be concerned about the image of public transit, and specifi c agencies, among elected offi - cials, voters, and potential transit customers. On the other hand, planners should also want to know about the criti- cisms and concerns aired via social media as a matter of democratic ethics and accountability, even if such ex- changes are, at times, adversarial and confrontational. Planners have for decades worked to create a strong mes- sage about how important transit is to cities and to the environment. Failing to attenuate, when possible, consis- tently negative commentary about transit or its patrons makes little sense if planners want people to consider transit use an amenity of urban living.

The evidence suggests that engagement online via interaction with individual commenters may pay off much more than blasting information, and those benefi ts can accrue both in content related to the agency’s reputation and to the planning dialog. If the planning profession exists, in part, to act as custodians of democratic dialog about the future of cities and the role that transit should play in cities, encouraging a positive presence on social media appears to be one way to foster better digital civitas.

Notes 1. One of the most high-profi le instances of such crossover occurred in 2008 when Jersey Shore reality TV personality Nicole “Snooki” Pelozzi, a person who—if her perpetually orange skin tone is any indicator—has a deep commitment to tanning services, tweeted her concern about a proposed tax on tanning beds. Presidential hopeful John McCain’s offi cial Twitter responded to her complaint by saying if he became president, he would never tax her tanning bed use. As trivial as it sounds, the exchange hit all the major network television news outlets. It was also covered on traditional news websites like The Huffi ngton Post

and The New York Times, and it enabled McCain’s campaign to move his most consistent message—his stance on taxes—to audiences that were not necessarily following his campaign or the GOP platform. 2. The Top 30 here simply refers to passenger volumes, not a quality ranking. 3. Available online at http://www.jdpower.com/consumer-ratings/travel/ ratings/909201528/2013-North+America+Airline+Satisfaction+Study/ index.htm 4. A “feed” is a data format that allows writers to syndicate information that is automatically “fed” to other users. With Twitter, to follow is to subscribe to a feed attached to another’s account. Following behavior is another potential way to interact. 5. Krippendorf ’s alpha is a statistical measure of the extent to which coders mark the same content consistently.

References Aday, S. (2010). Leading the charge: Media, elites, and the use of emotion in stimulating rally effects in wartime. Journal of Communica- tion. 60(3), 440–465. doi:10.1111/j.1460-2466.2010.01489.x Ajzen, I., & Fishbein, M. (2005). The infl uence of attitudes on behav- ior. In B. T. Johnson & P. Zanna (Eds.), The handbook of attitudes (pp. 173–222). Mahwah, NJ: Erlbaum. Ball-Rokeach, S. J., & DeFleur, M. L. (1976). A dependency model of mass-media effects. Communication Research, 3(1), 3–21. doi:10.1177/00936502760030010 Bay Area Rapid Transit (BART). (2014). Quarterly performance report Q1. Retrieved from http://www.bart.gov/sites/default/fi les/docs/QPR_ Report_FY2014-q4_FINAL.pdf Bregman, S. (2012, January). Uses of social media in public transportation: Summary of fi ndings from TCRP Synthesis SB-20. Paper presented at the Transportation Research Board 91st Annual Meeting, Washington, DC. Brenner, J. & Smith, A. (2013). 72% of online adults are social network- ing site users (Pew Internet Project). Retrieved from http://pewinternet. org/Reports/2013/social-networking-sites/Findings/Twitter.aspx Collins, C., Hasan, S., & Ukkusuri, S. (2012, January). A novel transit riders’ satisfaction metric: Riders’ sentiments measured from online social media data. Paper presented at the Transportation Research Board 91st Annual Meeting, Washington, DC. Drew, D., & Weaver, D. (1990). Media attention, media exposure, and media effects. Journalism & Mass Communication Quarterly, 67(4), 740–748. doi:10.1177/107769909006700428 Grönroos, C. (2000). Service management and marketing: A customer relationship management approach. New York, NY: John Wiley & Sons. Gunther, A. (1991). What we think others think: Cause and conse- quence in the third-person effect. Communication Research, 18(3), 355–372. doi:10.1177/009365091018003004 Hanson, S., & Johnston, I. (1985). Gender differences in work-trip length: Explanations and implications. Urban Geography, 6(3), 193–219. doi:10.2747/0272-3638.6.3.193 Hu, M., & Liu, B. (2004, August). Mining and summarizing customer reviews. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-2004, full paper), Seattle, WA. Los Angeles County Metropolitan Transportation Authority. (2014) Service performance analysis. Retrieved from http://isotp.metro.net/ Mai E., & Hranac, R. (2012, January). Twitter interactions as a data source for transportation incidents. Paper presented at the Transportation Research Board 92nd Annual Meeting, Washington, DC. Retrieved from http://www.cs.uml.edu/~hachreka/fi les/related/13-1636.pdf

RJPA_A_980439.indd 231RJPA_A_980439.indd 231 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

232 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Massachusetts Bay Transportation Authority. (2014). MBTA perfor- mance scorecard. Retrieved from http://www.mbta.com/uploadedfi les/ About_the_T/Score_Card/2014-8%20Scorecard.pdf Mitchell, A. & Page, D. (2013). Twitter news consumers: Young, mobile and educated (Pew Research Center, Pew Journalism Project). Retrieved from http://www.journalism.org/2013/11/04/twitter-news-consumers- young-mobile-and-educated/ Moss, M., & Kaufman, S. (2013). Effectiveness of social media among transportation providers in the New York City region (Final Report from the Rudin Transportation Center). Retrieved from http://www.utrc2. org/research/projects/social-media-among-transportation-providers- NYC National Transit Database. (2013). FY13 data. Retrieved from http:// www.ntdprogram.gov/ntdprogram/data.htm New York Metropolitan Transportation Authority. (2014). Perfor- mance dashboard. Retrieved from http://web.mta.info/persdashboard/ performance14.html Pang B., & Lee, L.. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135. Philips, L. E. (2010, December 28). Trends in consumers’ time spent with media. eMarketer. Retrieved from http://www.emarketer.com/ Article/Trends-Consumers-Time-Spent-with-Media/1008138 Potter, W. J. (2012). Media effects. Los Angeles, CA: Sage. Scheufele, D. A. (1999). Framing as a theory of media effects. Journal of Communication, 49(1), 103–122. doi:10.1111/j.1460-2466.1999. tb02784. South Coast British Columbia Transportation Authority. (2013). 2013 statutory annual report appendices. Retrieved from http://www. translink.ca/~/media/Documents/about_translink/governance_and_ board/board_minutes_and_reports/2014/march/2013_statutory_an- nual_report_appendices.ashx South Coast British Columbia Transportation Authority. (2014). 2014 statutory annual report. Retrieved from http://www.translink.ca/~/media/ documents/about_translink/corporate_overview/annual_reports/statutory_ annual_report/2013_statutory_annual_report_appendices.ashx Southeast Pennsylvania Transportation Authority. (2014). On-time performance report. Retrieved from http://www.septa.org/service/rail/otp. html Toronto Transit Commission. (2014a). Q1 quarterly reports. Retrieved from https://www.ttc.ca/PDF/Customer_Service/Quarterly_Reports/ Route_Performance%20_Q2%202014.pdf Toronto Transit Commission. (2014b). Ridership: Ridership numbers and revenues summary. Retrieved from http://www1.toronto.ca/wps/ portal/contentonly?vgnextoid=a3b7c87477438310VgnVCM1000003d d60f89RCRD Tri-Met. (2014). Performance dashboard. Retrieved from http://trimet. org/about/dashboard/index.htm Washington Metropolitan Area Transit Authority. (2014). ATA performance scorecard. Retrieved from https://wmata.com/about_metro/ scorecard/index.cfm Weinsten, A. (2000). Customer satisfaction among transit riders: How customers rank the relative importance of various service attributes. Transportation Research Record, 1735, 123–132. doi:10.3141/1735-15 Williams, K. (2006). A comparison of travel behaviors of African Ameri- can and White travelers to an urban destination: The case of New Orleans (Unpublished doctoral dissertation). University of New Orleans, New Orleans, LA. Retrieved from http://scholarworks.uno.edu/cgi/viewcon- tent.cgi?article=1453&context=td

Technical Appendix

Obtaining Twitter Data for Machine Coding Twitter is a private company, and it will shut searches

down if they detect individuals running queries that might overload the system. Twitter guidance for analysts and developers states that they can accommodate different searches of 1,500 tweets. In addition, Twitter does not retain an archive for searching. Their searches allow for three to seven days’ query on the API, depending on the number queried (Twitter, 2014). For large agencies like the Chicago Transit Authority (CTA), it is possible to get 1,500 tweets in one query. But for smaller agencies such as TriMet, the number of tweets likely to be returned in any given query may be much fewer. All tweets came from text collected during 20 one-day periods from March 2011 until May 2014 using the twitteR package for the open source software R (Gentry, 2011; R Development Core Team, 2011). The days were spaced throughout the years to capture different season’s effects on transit patron’s comments.

Data, Sampling, and Machine Learning Transit company sampling was derived from a listing

of the top 30 passenger-serving companies in the United States according to the passenger statistics from the Ameri- can Public Transportation Association. Also included in the population were all the transit companies that received the Outstanding Service Award (usually about four companies a year since 1983). From there, I drew a random sample of 10 companies. The random draw included two systems that yielded too few Twitter comments, even drawn over multiple years, for analysis. I replaced these two transit systems with the Toronto Transit Commission (TTC) and Translink in Vancouver (Canada) specifi cally because of their early adoption of social media strategies. Controls on the sample included additional random samples of celebri- ties drawn at random from the top 30 listings in the celeb- rity categories according to the Twitter’s propriety paid service, Twittercounter. Parks were drawn at random. The listing for public parks came from the Trust for Public Land’s listing of City Parks, from which 10 were randomly drawn from 163 parks listed (http://parkscore.tpl.org). Police departments were selected at random from the 50 largest metro areas and all the city police departments contained within U.S. metropolitan statistical areas (http:// www.policeone.com/law-enforcement-directory/). Social welfare terms and programs were nonrandomly selected due to their limited number.

RJPA_A_980439.indd 232RJPA_A_980439.indd 232 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 233

Machine Sentiment Scoring For this analysis, I take a basic counting approach

using an established lexicon of positive and negative Eng- lish and Spanish words developed by computer scientists (Hu & Liu, 2004) for businesses hoping to use text scraped from review pages like Yelp. Their lexicon comes via free distribution online at http://www.cs.uic.edu/~liub/FBS/ sentiment-analysis.html.

The algorithm developed for this manuscript was written using the open source software R. It matches text, assigns binary values, and summarizes as follows:

if x ∈ P ; = 1, else p = 0

if x ∈N; = 1, else n = 0

∑p − ∑n = s

∀x ∈ X

where X connects the entire collection of tweets and x represents strings of characters (words) matched against the positive (P ) and negative (N ) lexicon (Hu & Liu, 2004). The resulting summary scores (s) is simply the positive minus the negative count. The approach is somewhat rough, and it is likely that it will mis-score at least some of the comments. The goal is that the algorithm, applied to a large enough body of text data, will produce a useful summary, even if it has scored some parts of the data imperfectly. A similar method covers the Spanish corpus, following Batyrshin et al. (2012).

The results of an analysis of variance (ANOVA) exam- ines whether the test groups vary by sentiment (see Table A-1). Tukey’s honest signifi cant difference (HSD) is used to make a set of confi dence intervals on the differences between means with the family-wise probably associated of covering that range. The Tukey tests identify places where the vari- ance suggests differences between particular group pairs.

Communication Style and Potential Infl uence on Sentiment

This material concerns the sample subjected to human coding. After a meeting to determine the relevant codes, two students and I coded the same commentary separately, and then one coder reconciled the different codes at the end using all the work done by the other two coders. The team applied multiple codes to text to resolve issues regarding consistency among coders on race and class variables due to the signifi cant overlap of those two concepts. Words such as “ghetto,” for example, systematically split coders, as ghetto has both class and race connotations. The fi rst run with coders yielded a Krippendorf ’s alpha of 0.71, which is on the

marginal end of acceptable intercoder consistency, so I went through alone and recoded the material again with multiple codes to yield an alpha of .84 based on the consensus about words and codes derived from the group code process.

The research team coded Twitter users by type, as listed in Table A-2. Business and news media names were apparent. In other instances, I looked up the individual profi le on anybody tweeting more than three times in the validation sample from the machine-coding exercise. Table A-2 lists the guidelines for coding by user type.

Different types of users tweet about different things, and they tweet with different frequencies. The human-coded

Table A-1. Test group ANOVA and Tukey HSD results.

Sum Sq Mean Sq F value Pr(>F)

All comments

Test group 2,337 584.2 581.4 0.0000***

Residuals 50,240 1

Slurs excluded

Test group 2,101 525.4 526.7 0.0000***

Residuals 49,865 1

diff lwr upr p adj

All comments

Parks-airlines 0.5035 0.4648 0.5421 0.0000***

Police-parks –0.5932 –0.6319 –0.5546 0.0000***

Transit-parks –0.4858 –0.5245 –0.4472 0.0000***

Police-airlines –0.0898 –0.1285 –0.0511 0.0000***

Transit-airlines 0.1074 0.0687 0.1461 0.0000***

Welfare-parks –0.5471 –0.5858 –0.5084 0.0000***

Welfare-airlines 0.0461 0.0075 0.0848 0.0100**

Welfare-police –0.0436 –0.0823 –0.0050 0.0177*

Welfare-transit –0.0213 –0.0999 –0.0226 0.0215*

Transit-police 0.0176 –0.0210 0.0563 0.7256

Slurs excluded

Parks-airlines 0.5098 0.4713 0.5483 0.0000***

Police-parks –0.5465 –0.5851 –0.5080 0.0000***

Transit-parks –0.4921 –0.5306 –0.4535 0.0000***

Welfare-parks –0.4920 –0.5305 –0.4535 0.0000***

Police-airlines –0.0367 –0.0753 0.0018 0.0702Ψ

Welfare-airlines 0.0178 –0.0207 0.0563 0.7158

Transit-airlines 0.0177 –0.0208 0.0563 0.7181

Welfare-transit 0.0001 –0.0385 0.0386 1.0000

Notes: The difference in customer sentiment between airlines and transit disappears when slurs regarding patrons are excluded from the data (shown in bold). Ψsignifi cant at 0.1; *signifi cant at 0.05; **signifi cant at 0.01; ***signifi cant at 0.000.

RJPA_A_980439.indd 233RJPA_A_980439.indd 233 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

234 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Table A-2. User type codes and prevalence in the validation sample.

Group code Percentage Description

Agency — Tweets from the agency itself.

Watchdog/ satire

2% Individuals who tweet from satire accounts mocking the agency’s customer service feed or that highlight service or public management programs.

Elected 1% Politicians’ accounts. For example, individual mayors go here; tweets from the mayor’s offi ce are categorized as coming from “government” sources below.

Patron 61% Individuals who tweet representing their personal experiences and ideas.

Planner 2% Individuals who identify themselves as professional planners or professional writers/advocates who write about cities or transit, but who do from their perspective, not representing an agency or institution. Some can also be considered business accounts.

Government 1% Government offi ces’ twitter accounts.

Operator 1% Former or current bus and rail drivers who tweet.

Enthusiast 15% An enthusiast reveals in their profi le a personal affection and interest in transit, such “I like trains.”

Business 3% Accounts representing businesses with business names. Individual business owners are listed elsewhere depending on the rest of their profi le.

NGO 3% Accounts representing advocacy organizations or other nongovernmental, nonprofi t interests, such as the Congress for New Urbanism.

News media 12% Accounts from traditional news outlets, such as The Washington Post and NBC News, and online media sources such as Streetsblogs.

RJPA_A_980439.indd 234RJPA_A_980439.indd 234 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 235

Table A-3. ANOVA results for sentiment and agency interactivity.

df Sum Sq Mean Sq F value Pr(>F)

Sentiment

Interactivity 2.0 88.0 44.0 66.6 0.0000***

Residuals 5228.0 3456.0 0.7

Timing

Interactivity 2.0 2.0 1.0 5.0 0.0070**

Residuals 5228.0 1058.0 0.2

Driver conduct

Interactivity 2.0 0.5 0.2 3.8 0.0214*

Residuals 5228.0 307.7 0.1

Staff conduct ns ns ns ns ns

Facilities

Interactivity 2.0 9.8 4.9 22.7 0.000***

Residuals 5228.0 1126.7 0.2

Climate

Interactivity 2.0 0.2 0.1 5.3 0.0051**

Residuals 5228.0 101.7 0.0

Security ns ns ns ns ns

Age ns ns ns ns ns

Behavior

Interactivity 2.0 6.0 3.0 23.8 0.0000***

Residuals 5228.0 656.2 0.1

Class

Interactivity 2.0 0.0 0.0 3.5 0.0310*

Residuals 5228.0 33.8 0.0

Disability ns ns ns ns ns

Race

Interactivity 2.0 0.0 0.0 2.7 0.0643Ψ

Residuals 5228.0 42.8 0.0

Gender

Interactivity 2.0 0.1 0.0 3.9 0.0211*

Residuals 5228.0 38.8 0.0

LGBT ns ns ns ns ns

Size

Interactivity 2.0 0.1 0.0 8.4 0.0002***

Residuals 5228.0 20.9 0.0

Planning

Interactivity 2.0 4.2 2.1 18.9 0.0000***

Residuals 5228.0 582.8 0.1

Management

Interactivity 2.0 0.3 0.2 8.4 0.0002***

Residuals 5228.0 103.7 0.0 ψsignifi cant at 0.1; *signifi cant at 0.05; **signifi cant at 0.01; ***signifi cant at 0.000; ns = nonsignifi cant.

RJPA_A_980439.indd 235RJPA_A_980439.indd 235 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

236 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Table A-4. Tukey post-test results on factors by interaction level.

diff lwr upr p adj Score Low-high –0.3504 –0.4217 –0.2791 0.0000*** Medium-high –0.2188 –0.2908 –0.1467 0.0000*** Medium-low 0.1316 0.0725 0.1908 0.0000*** Timing Low-high –0.0510 –0.0904 –0.0116 0.0069** Medium-high –0.0230 –0.0628 0.0169 0.3679 Medium-low 0.0281 –0.0047 0.0608 0.1098 Driver conduct Low-high –0.0251 –0.0464 –0.0039 0.0155* Medium-high –0.0158 –0.0373 0.0057 0.1958 Medium-low 0.0093 –0.0083 0.0270 0.4320 Staff conduct ns ns ns ns Facilities Low-high –0.1164 –0.1571 –0.0757 0.0000*** Medium-high –0.0857 –0.1268 –0.0445 0.0000***

Medium-low 0.0308 –0.0030 0.0645 0.0828Ψ

Climate Low-high –0.0168 –0.0290 –0.0045 0.0038** Medium-high –0.0131 –0.0255 –0.0007 0.0346* Medium-low 0.0037 –0.0065 0.0138 0.6748 Security ns ns ns ns Age ns ns ns ns Behavior Low-high –0.0899 –0.1209 –0.0588 0.0000*** Medium-high –0.0471 –0.0785 –0.0157 0.0013** Medium-low 0.0427 0.0170 0.0685 0.0003*** Class Low-high –0.0079 –0.0150 –0.0009 0.0230* Medium-high –0.0050 –0.0122 0.0021 0.2225 Medium-low 0.0029 –0.0030 0.0087 0.4790 Disability ns ns ns ns Race

Low-high –0.0079 –0.0158 0.0000 0.0504Ψ

Medium-high –0.0055 –0.0135 0.0025 0.2385 Medium-low 0.0024 –0.0042 0.0090 0.6711 Gender Low-high –0.0079 –0.0155 –0.0004 0.0372* Medium-high –0.0021 –0.0097 0.0055 0.7934

Medium-low 0.0058 –0.0005 0.0121 0.0760Ψ

LGBT ns ns ns ns Low-high –0.0084 –0.0139 –0.0028 0.0011** Medium-high –0.0019 –0.0075 0.0037 0.6991 Medium-low 0.0065 0.0019 0.0111 0.0029** Planning Low-high –0.0750 –0.1043 –0.0457 0.0000*** Medium-high –0.0377 –0.0673 –0.0081 0.0080** Medium-low 0.0373 0.0130 0.0616 0.0009*** Management Low-high –0.0214 –0.0338 –0.0091 0.0001***

Medium-high –0.0124 –0.0249 0.0001 0.0514Ψ

Medium-low 0.0090 –0.0012 0.0192 0.0985Ψ

Ψsignifi cant at 0.1; *signifi cant at 0.05; **signifi cant at 0.01; **signifi cant at 0.000; ns = nonsignifi cant.

RJPA_A_980439.indd 236RJPA_A_980439.indd 236 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

Schweitzer: Planning and Social Media 237

Table A-5. ANOVA results for sentiment by user type.

df Sum Sq Mean Sq F value Pr(>F)

Score

Type 9 216 24.003 37.66 0.0000***

Residuals 5221 3328 0.637

Timing

Type 9 18.7 2.0761 10.41 0.0000***

Residuals 5221 1041.1 0.1994

Driver conduct

Type 9 1.59 0.17642 3.004 0.0014**

Residuals 5221 306.58 0.05872

Staff conduct ns ns ns ns ns

Facilities

Type 9 23.7 2.6353 12.37 0.0000***

Residuals 5221 1112.8 0.2131

Behavior

Type 9 23.8 2.6474 21.65 0.0000***

Residuals 5221 638.3 0.1223

Climate ns ns ns ns ns

Security ns ns ns ns ns

Age ns ns ns ns ns

Planning

Type 9 10 1.1134 10.07 0.0000***

Residuals 5221 576.9 0.1105

Management

Type 9 0.63 0.06952 3.51 0.0002***

Residuals 5221 103.41 0.01981

Planning by D-level and commenter type

Type 9 10 1.1134 10.104 0.0000***

Category 2 1.9 0.9251 8.396 0.0002***

Residuals 5219 575.1 0.1102

Management by D-level and commenter type

Type 9 0.63 0.06952 3.514 0.0002***

Category 2 0.15 0.07642 3.863 0.0211*

Residuals 5219 103.26 0.01978

*signifi cant at 0.05; **signifi cant at 0.01; ***signifi cant at 0.000; ns = nonsignifi cant.

RJPA_A_980439.indd 237RJPA_A_980439.indd 237 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15

238 Journal of the American Planning Association, Summer 2014, Vol. 80, No. 3

Table A-6. Tukey HSD results by user group.

diff lwr upr p adj

Planning

Operator-business –0.3122 –0.5504 –0.0739 0.0014*

Watchdog-business –0.1520 –0.2884 –0.0155 0.0156*

Operator-enthusiast –0.3250 –0.5476 –0.1024 0.0002**

Patron-enthusiast –0.0956 –0.1376 –0.0536 0.0000***

Watchdog-enthusiast –0.1648 –0.2716 –0.0580 0.0000***

Watchdog-gov –0.1805 –0.3540 –0.0070 0.0336*

Operator-news media –0.2768 –0.5003 –0.0534 0.0035**

Patron-news media –0.0474 –0.0936 –0.0013 0.0379*

Planner patron 0.0549 –0.0753 0.1851 0.9457

Watchdog-news media –0.1166 –0.2251 –0.0082 0.0235*

Operator-NGO –0.2684 –0.5052 –0.0315 0.0125*

Watchdog-planner –0.1715 –0.3301 –0.0130 0.0219*

Patron-operator 0.2294 0.0093 0.4496 0.0331*

Planner-operator 0.3317 0.0802 0.5833 0.0013**

Management

Planner-enthusiast –0.0026 –0.0571 0.0520 1.0000

Patron-news media –0.0203 –0.0399 –0.0008 0.0334*

Planning by D-level and commenter type

Patron-enthusiast –0.0229 –0.0407 –0.0051 0.0019**

Patron-news media –0.0203 –0.0398 –0.0008 0.0332*

Low-high –0.0462 –0.0753 –0.0171 0.0006**

Management by D-level and commenter type

Patron-enthusiast –0.0229 –0.0407 –0.0051 0.0019**

Patron-news media –0.0203 –0.0398 –0.0008 0.0332*

Low-high –0.0138 –0.0262 –0.0015 0.0232*

*signifi cant at 0.05; **signifi cant at 0.01; ***signifi cant at 0.000.

sample was drawn from the entire database according to a random sample stratifi ed for each agency by the proportion of each user type. ANOVA results, shown in Table A-3, show whether the sentiment scores exhibit a signifi cant difference between agencies according to dialog. The Tukey HSD posttest results follow in Table A-4. Table A-5 presents the ANOVA results on sentiment for each fi eld by user type, and Table A-6 presents the Tukey results for pairs.

References Batyrshin, I., González Mendoza, M., Sidorov, G., Miranda-Jiménez, S., Viveros-Jiménez, F., Gelbukh, A., … Gordon, J. (2013). Empirical

study of machine learning based approach for opinion mining in tweets. Lecture Notes in Computer Science: Advances in Artifi cial Intelligence, 7629, 1–14. doi:10.1007/978-3-642-37807-2 Gentry, J. (2011). twitteR: R based Twitter clien.(R package version 0.99.9). Retrieved from http://CRAN.R-project.org/package=twitteR Hu, M., & Liu, B. (2004, August). Mining and summarizing customer reviews. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-2004, full paper), Seattle, WA. R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/ Twitter. (2014). Using the Twitter API. Retrieved from https://dev. twitter.com/docs/using-search

RJPA_A_980439.indd 238RJPA_A_980439.indd 238 11/12/14 6:58 AM11/12/14 6:58 AM

D ow

nl oa

de d

by [

A ri

zo na

S ta

te U

ni ve

rs it

y] a

t 10

:4 5

13 F

eb ru

ar y

20 15