media regulation

Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=rdij20

Digital Journalism

ISSN: 2167-0811 (Print) 2167-082X (Online) Journal homepage: https://www.tandfonline.com/loi/rdij20

Public Service Chatbots: Automating Conversation with BBC News

Bronwyn Jones & Rhianne Jones

To cite this article: Bronwyn Jones & Rhianne Jones (2019): Public Service Chatbots: Automating Conversation with BBC News, Digital Journalism, DOI: 10.1080/21670811.2019.1609371

To link to this article: https://doi.org/10.1080/21670811.2019.1609371

Published online: 09 Jul 2019.

Submit your article to this journal

Article views: 80

View Crossmark data

Public Service Chatbots: Automating Conversation with BBC News

Bronwyn Jonesa and Rhianne Jonesb

aLiverpool, United Kingdom of Great Britain and Northern Ireland; bUniversity of Salford, Manchester, United Kingdom of Great Britain and Northern Ireland

ABSTRACT Automation of journalistic tasks is growing with the development of increasingly sophisticated software for newsgathering, production, and distribution. Bots are one form of algorithmic technology that has found a place in the modern newsroom, with chatbots leading the way as news organisations seek to attract new audiences using conversational forms of journalism. Recent advances in artificial intel- ligence (AI) and machine learning (ML) have fuelled increasing experimentation with machine autonomy and there has been much hyperbole in the press about the extent and impact of this on jour- nalism. Looking at on-the-ground trials in audience-facing bots at the UK’s largest public broadcaster, we find a significantly more restricted picture. News bots at The BBC to-date have been basic, do not use ML, and have rarely been integrated into news produc- tion. The organisation is laying groundwork for development of more interactive news formats with an increasingly conversational tone and individual mode of address as part of a strategy for increased personalisation, which is likely to involve growing levels of ML. In the process, bots are reconfiguring working practices and infrastructure, posing new editorial and technical challenges, and redefining relationships with audiences. We discuss the implications of this for public service media.

KEYWORDS Automated journalism; BBC; bots; chatbots; computational journalism; news bots; public service media

What’s a Chatbot? Let’s Talk

Following a rapid growth in the development of automated software agents known as bots since 2016, the news industry is increasingly exploring how they can be put to use for news production and distribution. Bots are just one of the many examples of automated software in journalism that are opening up opportunities for creating con- tent at scale, at previously unattainable speed, and in a customised or individually per- sonalised manner. They form part of what has been termed “robot,” “automated” and “algorithmic journalism” (Carlson 2015; Caswell and D€orr 2017; D€orr 2015; Montal and Reich 2017) alongside other computational approaches such as fact-checking (Graves 2018), which build on the foundations of computer-assisted reporting (Flew et al. 2012). Chatbots – a subset of bots that have caught the imagination of news publish- ers – are software programmes designed to converse with a human through natural

CONTACT Bronwyn Jones [email protected] � 2019 Informa UK Limited, trading as Taylor & Francis Group

DIGITAL JOURNALISM https://doi.org/10.1080/21670811.2019.1609371

language (Gorwa and Guilbeault 2018). To-date, most have been text-based, messen- ger-type interfaces, although innovation in voice interaction has grown recently (Barot 2017; Newman 2018, 34). This study illuminates the state of play in news bot innov- ation at the end of 2017 in the UK’s largest public service broadcaster, the BBC. It analyses how the research and development arm of the BBC has developed and employed bots, including chatbots, for news and the implications of this for public service journalism.

Chatbots are part of a wider group of Conversational User Interfaces (CUIs), which mimic everyday human dialogue in the form of a conversation, usually by employing informal and friendly language. They range from very basic and rule-based to more intel- ligent or “smart” bots driven by artificial intelligence (AI) and machine learning (ML) tech- nologies1 (e.g., Jones 2018). The most basic chatbots are automated (or semi-automated) but not autonomous – they play out only the functions that are inscribed into them by a programmer, using pre-scripted and pre-organised inputs. They reply to humans by traversing pre-coded decision trees using rules, thus simulating interaction but never actually deviating from pre-scripted narrative structures. A familiar sight in customer ser- vice, they are also now common in news and information distribution (Lokot and Diakopoulos 2015). ML-driven chatbots are, however, becoming more widespread. Most use systems that convert unstructured human input (speech, text, or gestures) into machine-readable form, apply Natural Language Processing (NLP), and translate the out- put using Natural Language Generation (NLG) to mimic human speech/text. An oft-cited example of an ML-powered chatbot is Microsoft’s Tay, a Twitter bot which learnt from users’ interactions and became notable as much for going awry as for its innovative technology – after it was shut down because of obscene and inflammatory tweets (Neff and Nagy 2016). Chatbots are, however, just one example of a much wider set of auto- mated software agents known as bots, a term derived from “robots.” Fundamentally, a bot is a software programme that completes an automated task. The term has become a buzzword in the news industry (Barot 2015), and AI- and ML-driven bots are only now beginning to play a part in news gathering, production, and distribution (Lokot and Diakopoulos 2015; Thurman, D€orr, and Kunert 2017). They are part of a broader growth in computational journalism (Flew et al. 2012) using digital and networked technology in the newsroom and fit within a recent turn to data (Coddington 2015).

As powerful mediators of social interaction online, social media platforms such as Twitter, Facebook and Instagram have become hubs of activity for people developing and deploying bots for various purposes. Scholars have looked at the implications for sociality of the relationship between humans and these “software processes that are programmed to appear to be human-generated within the context of social network- ing sites” (Gehl and Bakardjieva 2016, 2). They have explored use of such bots for news and information dissemination (Lokot and Diakopoulos 2015) but also for public opinion manipulation and computational propaganda (Woolley and Howard 2017), including spamming, harassment, and falsifying trends and consensus. Lokot and Diakopoulos argue that in paying attention to the more nefarious uses of bots, critics have overlooked the “potentially positive and beneficial utility of automated news and information sharing,” including how bots may “contribute to positive effects in the public media sphere if employed ethically and conscientiously” (2015: 3).

2 B. JONES AND R. JONES

It is clear that bot development has been uneven across the news industry, and unhelpful hyperbole about not only the risks and dangers but also the benefits of bots has been characteristic of the media attention they garner. Nuanced analysis of how news production is being reshaped by the shift toward computational and algorithmic journalism is therefore vital (Anderson 2012), and any such analysis must recognise the co-construction of technology and society and approach technologies as complex, socio- material phenomena (Gillespie, Boczkowski, and Foot 2014). What this requires is a rec- ognition that technologies are not neutral and that they exert influence through materi- ality but are also enacted in material-discursive practices (Bucher 2018) – a process of mutual shaping in situated practice. In the case of developing new technology for jour- nalism, the shaping of what is desirable for the technology, and of the conditions that determine what is possible for the technology, is being done by engineers and develop- ers alongside journalists and managers, who all bring values and priorities from their specific organisational and cultural backgrounds. As Lewis and Westlund argue, a socio- technical emphasis in the study of institutional news production, which takes into con- sideration “the full range of actors, actants, and audiences engaged in cross-media news work activities” (2015: 19) is in order. In this vein, a body of research has honed in on increasing collaboration between technologists and journalists (Nielsen 2012; Lewis and Usher 2013, 2014, 2016). Lewis and Usher foreground how programmers and their ethics are assuming a greater role in the journalistic field (2016) but also how the fusion between the social worlds of journalism and technology requires “significant, coordi- nated, and sustained effort” as the “barriers between each field’s understanding of the other are real” (2014: 9). Bots, then, should not be viewed as technologies separate from their contexts of production and use. They are best analysed by locating them in – or conceiving of them as – a socio-technical assemblage that includes hardware, software, techniques and practices but also guidelines, goals and values.

Underpinned by this conceptual approach, this paper hones in on bots in the pub- lic service news sector using the BBC as a case study to explore the current state of play and potential implications. Bots raise specific questions for public service media (PSM) (Sørenson and Hutchinson 2018). The BBC is an important case study site as it is the UK’s largest broadcast news organisation and a public service broadcaster with a remit to develop technology for the public good. Furthermore, its secure funding model underpins a comparatively well-resourced research and development depart- ment that is not bound by a commercial agenda and has a long history of innovating in media technologies in the public interest. The BBC is understandably highly focussed on the work of creating these new computational tools for the newsroom and this paper takes a much-needed step back from the day-to-day practical concerns with usability to critically assess these attempts.

Related Work

News Bots: Automating the Production Process

Bots have existed for almost as long as computers, and the term could technically encompass a diverse array of applications and programmes employed by newsrooms. However, it is more commonly used to describe software agents with a recognisable

DIGITAL JOURNALISM 3

audience interface through which an audience/user interacts – not those agents that work solely behind the scenes. Some bots are journalist-facing and others audience- facing, though they can be both. Early bots often performed singular tasks, but many bots, or more accurately assemblages (DeLanda 2006) of bots, perform numerous interlinked tasks and straddle these categories. Their role in the production process can be further broken down according to the functions they perform: news gathering, production and distribution.

Newsgathering bots search, monitor, retrieve, alert or nudge. For example, BuzzFeed’s Buzzbot for Facebook Messenger, which collected photos and stories from users at a Republican Convention in 2016, WNYC’s bot that monitored federal court documents for updates, and the Associated Press bot that tracks data breaches (You 2015). These bots may flag up anomalies they find in data so journalists can investi- gate, or, in the case of those being developed at the Duke Reporters Lab, they may identify and suggest claims to be fact-checked (McKinney 2018). Production bots col- late, analyse, create, edit, or visualise and, increasingly, they automate news writing. An early and oft-cited example of this was the LA Times Quakebot developed in 2014 to monitor earthquake magnitude and automatically write up reports before emailing an editor to alert them – both gathering and producing news. More recently, bots such as the Washington Post’s Heliograph use templates written by humans combined with a source of structured data to construct stories (Keohane 2017). There has been significant investment in this area by, amongst others, Google, which in 2017 granted the UK’s Press Association news agency £622,000 for its Reporters and Data and Robots (Radar) project (Ponsford 2017). Finally, distribution bots (re-)publish/broadcast, share and respond. For example, automated live tweeting of information from struc- tured data sets by bots is commonplace around the world, e.g., for cricket games, local air quality alerts, and election results at the Hindustan Times (Wang 2017). Bots have also widely been used by both legacy organisations such as the Washington Post and digitally native providers like Quartz for news recommender systems to sug- gest content based on user preferences or activity.

Importantly, distribution is where chatbots and other CUIs have been most com- monly applied. They are often designed with the goal of improving audience inter- action and engagement with news organisations and their content by creating conversational formats that deviate from the more static, traditional modes commonly associated with journalism. For example, the New York Times Politics Bot for Facebook Messenger aimed to “combine the intimacy and charm of a human with the utility of a bot” during the Trump campaign (Phelps 2017). It sent out daily alerts with election forecasts but also hosted conversations every morning scripted by one of their polit- ical reporters, with which 25,000 people interacted (ibid.).

Conversational Journalism and the Public Service Context

News organisations, including the BBC, are responding to fragmenting audiences and rising competition from mobile, social, and digital media (Picard 2010; Anderson 2012). Whether the aim is the commercial one of more advertising revenue or the democratic one of an informed citizenry and a healthy public sphere, both commercial

4 B. JONES AND R. JONES

and public service news organisations are under pressure to reach new and under- served audiences. Like many PSM, the BBC hopes forms of digital innovation will help attract younger audiences whose changing consumption patterns reveal a shift to online and on-demand viewing (van Es 2017) and mobile and social media platforms. Sehl, Cornia, and Nielsen (2016) pinpoint three core interrelated challenges for PSM: 1) retaining younger audiences, 2) the shift to personal and mobile media and, 3) devel- oping effective ways of delivering public service news via third-party platforms includ- ing search engines, social media, video-hosting sites, and messaging apps. These challenges have pushed PSM to publish on off-site platforms such as social media and messaging apps (ibid.) using CUIs designed to make the news more engaging and better suited to these platforms.

News chatbots exemplify a rise in “conversational journalism” approaches that aim to achieve just this by engaging people who are traditionally not users of news through the leveraging of new technologies to create formats that are informal, interactive and novel. The BBC is aiming to cultivate a youth market for “accurate but accessible” news, stating: “Our belief is that a less formal and more conversational style will be less off put- ting to younger readers” (BBC News Labs 2018). The ambition to create new forms of dialogue with news audiences pre-dates CUIs and has roots in broader changes to the relationship between audiences and journalism in an Internet age characterised by digital technologies that “shift the direction of communication from a one-to-many broadcast- ing system to a many-to-many conversational system” (Anderson 2011, 532). Recent developments in Human-Machine Communication (HMC) (see Guzman 2018; and Lewis, Guzman and Schmidt 2019) usefully move beyond conceptualising communication “as a human process through machines” to focus on “the creation of meaning among humans and machines” (Guzman 2018: 1).

The growth in news bots is also happening within larger economic processes shaping the journalism industry. Technologies like bots, which promise efficiency savings and can demonstrate increased engagement of younger audiences, resonate in public service newsrooms facing budget reductions, and they work to allay fears of declining relevance amongst future generations. Bots also represent shifts in newsroom culture, particularly towards personalisation (Helberger 2015) and the increasing measurement and tracking of audiences (Carlson 2018). The move to algorithmic personalisation in public service journalism has sparked debates focussed on either the risk it may pose to PSM values like universality, access and diversity (Bennet 2018, Sørenson and Hutchinson 2018) or the opportunities it may bring to promote diversity of supply and stimulate diversity exposure (Helberger 2015). For Sørenson and Hutchinson (2018) the five main challenges PSM face are: 1) balancing popularity and distinctiveness, 2) diversity of exposure to pro- gramming, 3) transparency of the logic underlying recommendations, 4) user sovereignty and, 5) the issue of dependence on, or independence from, commercial intermediaries. Such debates foreground difficulties in this context for the PSM “mandate to serve a full citizenry of a diverse nation while simultaneously creating common culture” (Lotz 2018, 47), facilitated through the shared media experience (Scannell 2005). Helberger argues that PSM today are “at a crossroad where they must decide how personal, persuasive, and responsive their relationship to the audience should be, and what safeguards are needed to preserve autonomy, privacy, and the public sphere” (2015: 1325).

DIGITAL JOURNALISM 5

Research Questions and Methods

This paper asks three research questions:

1. How is the BBC experimenting with bots in news? 2. What are the implications for news production and distribution? 3. What impact might this have on public service journalism?

It answers these questions by compiling a case study of news bots at the BBC, including those actually deployed into news products and those in development and testing. Importantly, we employ a mutual shaping conceptual framework that views technologies as socio-material artefacts that are constructed in situated practice. We use this approach to analyse journalists’ and technologists’ accounts of bots during the crucial development and testing period, during which meaning is being inscribed into, and ascribed to, the technology and the technology begins influencing, and being influenced by, journalistic practice. It is important to create site-specific under- standing of how computational journalism is developing within particular organisa- tions and sectors of the industry, and PSM are an underdeveloped area of study. As Young and Hermida point out, analysing how this technological change “combines with and emerges out of existing norms, routines, relationships, and social and mater- ial contexts,” we can “discern how digital media both constitute and are constituted by practice and innovation” (2014: 381).

Initial scoping research by the authors, who are a BBC journalist and senior technol- ogy researcher2 indicated the BBC had in total developed eleven bots – eight in-house and three outsourced to external companies. We chose to focus on the eight in-house bots, which were created by units within the BBC’s Research and Development (BBC R&D) department – seven by BBC News Labs (charged with driving innovation for BBC News) and one by Connected Studio (which brings wider networks together to gener- ate ideas for future technology).

We propose an analytical approach that interrogates how key decision-makers and actors conceive of and describe technologies, how they document and evaluate them, and how they create and appropriate narratives about them. Accordingly, we used two methods – semi-structured qualitative interviews and document analysis – and employed an iterative research process for thematic analysis based on both inductive and deductive approaches. Purposive sampling was used to identify interviewees, with the expert knowledge of the authors who work in the research context being lever- aged to make a situational judgement about which individuals should be selected. Through this process we identified twelve expert interviewees. They were key actors involved in technical development (3) and in editorial production (5), and leads/strate- gists/managers (4), who were able to place developments in the context of broader work in voice, machine learning and the BBC’s personalisation strategy. During the semi-structured interviews, conducted between October 2017 and February 2018, we asked questions relating to five areas formulated in reference to the literature, our analytical framework, and our research questions. These focussed on critical points in the life-cycle of the technology in order to draw out what influenced the material and social construction of the bots. They were: motivation for developing the bots;

6 B. JONES AND R. JONES

technical capabilities; process of development; editorial and ethical challenges; and cri- teria used to evaluate and measure success.

We also conducted analysis of internal documentation, including project outlines, progress reports, and evaluations, access to which was provided by interviewees both before and after interviewing. This was supplemented with publicly accessible informa- tion, including press releases and blogs. An iterative research process allowed us to identify a sample of relevant documents from those provided and bring them together with those we had independently identified. We then coded the interview data and documentation in two stages, firstly according to the five themes previously identified and then subsequently for other emerging themes. We triangulated the data and cross-referenced analysis from each method to seek validation of findings through corroboration (Denzin 1970). Using the data from these two methods, we developed case study accounts of each bot and highlighted six themes that we situate in the organisational context and that also address wider debates in the practitioner and academic community.

We begin by providing an overview of BBC news bots and then outline six themes through which this technology can be understood in the public service context. Finally, based on that analysis, we outline future opportunities and challenges regard- ing CUIs in public service news at the BBC and more broadly.

Findings

Overview of Bots

BBC R&D, led by BBC News Labs, developed and ran live versions of eight bots between 2015 and 20173 primarily for social media sites and messenger services. Grouped by platform, four were on Twitter, two on Facebook, one on Telegram and one within articles on the BBC website. Five of the eight trials were described as chat- bots, which engaged in some level of interactive, two-way dialogue or used conversa- tional tone for chat-like interactions with users, while the other three were simply one-way distribution bots publishing information. The forms of interaction designed into the five chatbots were varied, including a Q&A, a quiz, a subscription service for push notifications and a news summary service. Most of the bots re-used previously vetted and published BBC content with a small amount of re-formatting, but two bots involved tailoring or writing bespoke material.

Third parties also developed three other bots for BBC News, namely the BBC Politics Brexit bot and 2017 General Election bot for Facebook Messenger and the NewsChatta bot for WeChat. This paper concentrates on the eight bots developed in- house but also discusses issues raised by outsourcing bot development to third parties and the emerging organisational strategy for mitigating associated risks.

The findings are organised into six overarching themes: 1) Terminology and levels of intelligence, 2) Conversational formats, 3) Working practices and organisational structures, 4) Locating automation and the journalist-in-the-loop, 5) Evaluation and measures of success, 6) Third parties and outsourcing. These themes illuminate ways in which the journalists and technologists are seeking to make sense of bots and apply them for public service news work, shedding light on how they understand the

DIGITAL JOURNALISM 7

technology, act upon the possibilities it offers, and frame the decisions they make. Theme one shows that definitions of technology had yet to stabilise or formalise. Theme two illustrates how wider changes across media from one- to two-way inter- action and in favour of individualised and personalised formats were being employed by the organisation to justify the benefits of developing bots in line with the popular conversational tone prevalent on social media. It indicates that a shared organisational narrative – voiced by both technologists and journalists – was being marshalled around the value of bots in PSM for improving reach and engagement, particularly with young people. Theme three describes how technologies under construction inter- play with existing professional practice. Theme four indicates that the gains bots enable through automating news delivery are valued by technologists and journalists alike but by the latter only if they do not come at the expense of editorial control and oversight. Meanwhile, theme five explores how limited evaluation methods made it difficult to gauge public service value despite this being a stated aim. Finally, theme six highlights the significance of involving non-PSM actors in the construction of tech- nology and explicates the risks of outsourcing bots.

Terminology and Levels of Intelligence

Definitional boundaries had not yet stabilised and understandings were still being contested and under construction. Initial analysis revealed that no clear common def- inition of a bot was used consistently across the organisation and that a variety of terms were used interchangeably to denote an automated software agent, including chatbot, conversational user interface, agent, algorithm and skill. Gorwa and Guilbeault (2018) also recognise this lack of definitional clarity and foreground the problems it generates for industry, academia and policy. BBC developers described fol- lowing the naming conventions of the platforms for which they were developing soft- ware. For example, when working with Google’s Alexa, they termed what could otherwise be referred to as a bot as a “skill.” This convention for inheriting proprietary platforms’ descriptions was prevalent among both technologists and editorial staff and framed how they thought about the technologies with which they were working. One factor that became clear, however, was that only bots that were text-based and audi- ence-facing were consistently given the moniker “bot.” It was these that were chosen for analysis.

Among the eight bots identified (see Table 1), the first thing to note is that they did not use AI, ML, text-to-speech technology or NLG/NLP. Their level of “intelligent” conversation was therefore limited. They were very basic and relied on decision-tree structures to underpin their automated activity. One project lead described them as “small, lightweight, decision-tree” bots for which the content was curated by journal- ists and compared them to “a choose your own adventure story” (I-10). They were used primarily for content distribution and to gather material by searching internal pre-prepared sources, i.e., existing online articles, pre-prepared script, or internal data. There was in fact very limited modification of existing content. In these trials, the Twitter bots (UK Election 2015, EU Referendum 2016, US Election 2016 and UK Election 2017) functioned solely as an alternative distribution platform. The Telegram

8 B. JONES AND R. JONES

Ta bl e 1.

Ke y fe at ur es

an d fu nc tio

ns of

bo t ca se

st ud

ie s.

Pl at fo rm

(B BC

O nl in e, So ci al

M ed ia ,M

es se ng

er )

N ew

s Fu nc tio

n (A ll fo rm

s of

di st rib

ut io n)

So ur ce

M at er ia l( BB

C, no

n- BB

C)

Ch at bo

t? (In

te ra ct iv e/

co nv er sa tio

na lm

od e

of ad dr es s)

N ew

sb ot

BB C O nl in e

A ch at bo

t pl ac ed

at th e en d of

se le ct

on lin e ar tic le s th at

fo llo w ed

a m ul tip

le -c ho

ic e qu

es tio

n an d an sw

er fo rm

at us in g pr e-

sc rip

te d m at er ia l.

BB C

Ye s

M un

do Bo

t So ci al /M

es se ng

er (F ac eb oo k)

Su bs cr ip tio

n se rv ic e. Au

to m at ed

pu sh

up da te s tw ic e da ily ,l in ki ng

to BB

C Sp an is h la ng

ua ge

ne w s si te ,B

BC M un

do .M

an ua lp

us h of

br ea ki ng

ne w s to

su bs cr ib er s.

BB C

Ye s

Q ui z Bo

t So ci al /M

es se ng

er (F ac eb oo k)

A qu

iz ch at bo

t ru n on

Fa ce bo

ok an d in

an on

lin e ar tic le

us in g pr e-

sc rip

te d m at er ia l.

BB C

Ye s

U K El ec ti on

Bo t 20 15

So ci al (T w itt er )

Al lo w ed

us er s to

gi ve

th e fir st

ha lf of

th ei r po

st co de

in or de r to

pr o-

vi de

th e cl os es t m at ch in g co ns tit ue nc y re su lt an d a lin k to

in fo rm

a- tio

n on

th e BB

C w eb si te .

BB C

Ye s

EU Re

f Bo

t 20 16

So ci al (T w itt er )

Tw ee te d ou

t EU

Re fe re nd

um re su lts

fo r ea ch

co un

tin g ar ea

by ga th er -

in g re su lts

in re al -t im e fr om

BB C in -h ou

se sy st em

an d co m bi ni ng

w ith

TV gr ap hi cs .

BB C

N o

U S El ec ti on

Bo t 20 16

So ci al (T w itt er )

As ab ov e bu

t ad ap te d fo r th e 20 16

U S El ec tio

n (a ls o tw ee te d

in Sp an is h) .

BB C

N o

U K El ec ti on

Bo t 20 17

So ci al (T w itt er )

As ab ov e bu

t ad ap te d fo r th e 20 17

U K El ec tio

n. BB

C N o

BB C U zb ek

Bo t

M es se ng

er (T el eg ra m )

Su bs cr ip tio

n se rv ic e. U se s ex is tin

g RS S fe ed s to

se nd

la te st

he ad lin es /

su m m ar ie s/ en tir e ar tic le s in

a pr iv at e m es sa ge

as a w ay

to ge t

ne w s in to

a co un

tr y w he re

th e BB

C is bl oc ke d.

BB C

Ye s

DIGITAL JOURNALISM 9

Uzbek bot summarised and distributed existing BBC articles, while the Facebook Messenger BBC Mundo bot enabled subscription to push notifications. Only the Facebook Messenger quiz format and in-article chatbots necessitated re-wording text and creating connections between pieces of content to pre-format potential pathways through the non-linear text.

This basic functionality was recognised by the News Labs team who described their work as scratching the surface of what could be achieved. When making the BBC Mundo bot, they started with the premise that users “don’t always know how to inter- act” with an AI-powered smart bot and cited user “concern over black box algorithms and biased distribution platforms” as a motivation for giving users “clear, simple com- mands to control the news they receive” (He 2016). Journalists similarly pointed to the bots’ limitations, explaining, for example: “It doesn’t have AI – it doesn’t learn… it just has selective answers, for example saying ‘sorry, I don’t understand’ etcetera” (I-2), and “It was an early try” and “was a bit clunky” (I-1). Some expressed a desire for higher functionality, including a “need to improve communication skills, be clever, give more possibilities… I want to improve it, give the user more options.” (I-2). This limited functionality is not dissimilar to that of chatbots developed by other PSM, such as Australia’s ABC Newsbot.

It is clear that AI/ML-driven bots have not yet been incorporated into BBC News and that a cautious approach to bot capabilities has been adopted but that the organ- isation is learning from the challenges faced during these experiments to refine the format with a view to applying ML technologies. Prototypes built for hackathons have been more ambitious, for example using text-to-speech and experimenting with multi- lingual API services, but these had not been trialled for BBC News.

Conversational Formats

The main area of innovation was in chatbots, which exemplify a move towards more individualised and personalised formats in news by way of user interfaces designed to mimic conversation. The five chatbot trials reveal experimentation with new technical capabilities and editorial approaches with the explicit goal of creating a more personal tone for news and an individual mode of address, aimed at increasing reach and engaging less traditional news audiences, particularly the young. BBC developers intended chatbots to be more conversational, personal, and interactive – departing from traditional linear and formal modes of news presentation and distribution. The experiments were used to “measure public enthusiasm for chat-like interactions” (Document 1). One project lead said: “there’s been a convergence around personalisa- tion, customisation, and interaction” (I-10). Journalists conceived of the conversational approach as a means to an end – to get more people viewing their work. One explained: “I think we need to experiment with new ways of telling stories and if it’s allowing you to explain quite technical things in a more accessible way, that’s clearly important for our job” (I-1), whilst another said: “It’s a novelty… it goes to another platform that is quite shareable so content can potentially reach many people” (I-2). This indicates a shared organisational narrative around the value of bots in PSM for improving reach and engagement.

10 B. JONES AND R. JONES

The descriptor “conversational” was used by developers rather than journalists to describe two interlinked elements of the bots – the tone of voice that deviated from formal news presentation and the interactive back-and-forth through which the bot would respond to user prompts. The Q&A and quiz chatbots exemplify hybridised news formats whereby a tried and tested format is adapted to fit the capabilities, style and tone of social media and/or messenger apps. Even though the Q&A was placed inside conventional BBC online stories, it was designed to feel like a conversation instead of a linear explainer, sidebar, or fact box, greeting users with an informal: “Hello, I’m an experiment from BBC News Labs.” Only the Uzbek bot diverged from this design goal, driven as it was by the need to provide BBC news to an audience who were blocked from accessing it by national authorities. However, it too adopts a deliberately conversational mode of address consistent with the Telegram messaging platform for which it is built. Whilst not mainstream, the bots sit within the BBC’s push towards becoming a “pioneer” of personalisation with a publicly stated ambition to provide a “more personal and relevant” BBC that is more “about you” (Hall 2017). This only recently became viable through the move to a sign-in model, marking a sig- nificant change in the BBC’s relationship with its audience. BBC news chatbots cur- rently tailor the information they provide to users in a way more akin to explicit customisation than implicit personalisation (Zuiderveen Borgesius et al. 2016, 3). At the stage of development encountered during this study, the level of personalisation was extremely limited. For example, the BBC Mundo bot addressed users by name and varied the expressions used in messages to make the messages seem more like human conversation but then served each subscriber the same selection of news, working in a way that resembled the traditional broadcast function despite being accessed through a personal device. Though viewed by developers and journalists as a more interactive way to deliver news, the bots, in some circumstances, have new forms of passivity built into their structure. For example, the BBC Mundo bot and the Uzbek bot push information to users, removing any need to search it out, which might preclude serendipitous content discovery.

The BBC is re-thinking and redefining how audiences/publics can engage with pub- lic service news. Chatbots were designed specifically to foster new relationships that feel more like an informal one-to-one dialogue as opposed to a more formal public broadcast. As one journalist said: “the more friendly, the better” (I-1). Early research indicated that more people engaged with a bot when it was presented in the form of a person as “an expert” rather than as the organisation or simply a bot (Document 1). Chatbots on social media and messenger services blur the boundaries between public spheres of news and private, personal interactions. One project lead said private mes- senger services have fostered an environment where “you expect people to talk back to you” (I-10). In doing so, they make public news consumption akin to a personal and private experience, by design. This feeds into wider concerns around the collapse of public and private boundaries brought about by social media (Baym and boyd 2012) and around algorithmic personalisation in public service media (Bennet 2018; Helberger 2015; Sørenson and Hutchinson 2018). A personal message from the BBC arriving in the user’s Facebook inbox is notably different to the user landing on the same BBC website index pages as other people and being exposed to the wider

DIGITAL JOURNALISM 11

content on those pages. PSM understandably want to leverage the new opportunities offered by social media but also need to be cautious of ways platform logics might compromise public value (van Dyke and Poell 2015, 149). Developers recognised potential conflicts in making journalism conversational “while maintaining trust – off platform and on” (I-6).

Working Practices and Organisational Structures

The case studies show new forms of collaborative working between editorial and tech- nical teams – a trend found more widely across the news media industry (Holton and Belair-Gagnon 2018; Nielsen 2012; Lewis and Usher 2013, 2014, 2016). Both editorial and technical teams commissioned the bots, however, there was a clear partitioning of responsibilities between developer and journalist, and output remained under edi- torial control, revealing how bot technology interacts with existing structures of pro- fessional practice. Editorial teams proposed the idea for six of the bots, while the two Facebook Messenger bots were suggested by developers, indicating that both editorial needs or desires and technical opportunities are driving bot innovation. Particularly in the early stages, developers and journalists would work together closely – uncommon in the newsroom. For example, the reporter who worked with the first Twitter bot, and called it “quite hacky” and “improvised,” described how he sat up all night through the election: “It was me, a developer, and a laptop… If there are editorial changes, I can make them or if anything breaks down, the developer can rewrite the programme” (I-4).

Whilst working on these bots, journalists had to learn new skills and either take time out of their usual working routines or modify them. Changing work practices around the introduction of new technologies is not uncommon in news as journalists are regularly required to revise their skills in the light of institutional, economic, and technological change (Willnat, Weaver, and Choi 2013). As Powers notes, although evolving journalistic work processes are not new, they often force “new tasks on reporters and editors alike” (2012: 27). New tasks in these experiments included, for example, monitoring the performance of a live bot, responding to audience enquiries, and writing into new production software. Studies looking at more technically advanced conversational technology, such as natural language generation, in journal- ism (D€orr 2015) have pointed to journalists “migrating from a direct to an indirect role” (Napoli 2014, 350). Journalists working with the BBC bots continued to play a key role in the editorial and creative process, though work with bots remained only a small part of their job.

Journalists reported altered workflows – particularly having to set aside time to learn how to set up and work with the bots – but some considered this a part of their role and weighed it in relation to the benefits the bot brings. For example, the Mundo Facebook Messenger bot journalist noted an increase in his work – “It was a lot of work, I had to develop different scenarios, work iteratively with developers” – remark- ing on the process of “sending, reviewing and amending content” alongside technical development (I-2). This is in part because chatbots impose different requirements to traditional editorial creative practice, as a more modular form of content is needed to

12 B. JONES AND R. JONES

fit the technical format. Though the content creation work is not entirely new (e.g., Q&As already exist in linear formats in news), content needs to be further transformed to support the appearance of an unfolding dialogue. In order to get journalists on board, developers produced guidelines and advice for non-linear storytelling, includ- ing, for example, the importance of allowing the user to “stop at any point” and of leaving “no dead ends” (I-10). This exemplifies the impact of computational forms of journalism which necessitate content that can be easily queried, traversed and re- assembled according to computational logics as part of the trend to individualise and personalise media. However, bots were not always seen as placing an increasing demand on time. The Newsbot journalist said the content for the bot “took the same, if not less, time to create” as “the Q&As the BBC already runs” and was “more engaging” (I-1). A project lead explained that bot testing has led to “informal groups of journalists interested in bots now,” and new positions have been created that fuse editorial and technical skills, such as a Senior Journalist role as Bot Development Producer.

Locating Automation and the Journalist-in-the-Loop

Bots were valued by technologists and journalists alike for their role in distributing news but journalists were clear that this should not come at the expense of editorial control and oversight of content. The role of the bots in the news production process was limited to distribution and a small amount of processing (i.e., re-formatting). The gathering of information and initial creation of news remained in the hands of the journalist. Moreover, only two bots had what can be described as bespoke material made for them. The more creative part of the journalistic process, i.e., the knowledge work, therefore remained in the purview of the human. One project lead explained that “it [the content and bot script] is all curated. All written by journalists.” Journalists additionally felt the need to maintain oversight over the published product. For instance, the editorial lead of the Twitter EU Referendum bot remarked sitting up all night to “watch the updates go live” in order to monitor output and “work out what to do when there was a mistake in terms of corrections” (I-4). This suggests that with the benefits of scalability, i.e., the ability to increase the scale of the audience, comes a desire to maintain editorial control and ensure the automated tasks performed by the bot are subject to quality control. It highlights the perceived importance, particu- larly as automation in bots increases, of having a “journalist-in-the-loop.”

When journalists were asked about their views on more technically sophisticated conversational bots, they acknowledged this needed “more thought” and expressed concerns about “accountability” and “editorial balance,” and the need “to get the bal- ance of automation and curation right” (I-1). The in-article Newsbot journalist, for example, warned that if the bot was to be used more widely or on topics with polar- ised views such as politics, then “we should spend time thinking about which ques- tions are put first” and therefore most likely to be accessed, and asked, “are you offering a balance?” (I-1), adding that he thinks it is crucial to always have editorial oversight. If chatbots with more algorithmic complexity are developed to tell a full news story (e.g., in the form of a “conversation” through a messaging app), they

DIGITAL JOURNALISM 13

would have to ensure that the core elements needed for an editorially robust and bal- anced report are presented to all users despite varying user journeys. Furthermore, this suggests that if chatbots become more common, changes to workflow and poten- tially the creation of new roles may be required for oversight. This was a point made by BBC technology strategists who were thinking about how to manage the step change from small scale and ad hoc experiments to what they describe as a more scalable, consistent and resilient approach that would support an integrated chatbot ecosystem. This finding exemplifies a challenge for public service journalism: preserv- ing control over the final outputs of an editorial process (as was possible with a stand- ardised broadcast) when there are multiple variations (afforded by personalised news, enabled by automation).

As conversations become more “intelligent” (e.g., employing NLP and capable of becoming more implicitly and highly personalised), issues around balance, universality, diversity, accountability and transparency etc. are likely to become more pressing and pronounced, while at the same time there is a risk of their becoming more concealed (e.g., behind algorithmic decision-making or voice interfaces). The amalgamation of algorithmic processes, audience data and semantic markup of content that enables personalised or responsive conversations will take place behind an interface, which, unless due process is in place, will make scrutiny of when, why and to whom content is shown increasingly difficult. The BBC has been discussing these issues broadly across its application of ML (O’Donnell 2017; BBC 2017).

Evaluation and Measures of Success: What Makes a Good Bot?

Evaluation during trial periods of prototype technologies plays an important role in justifying their continued development and use. However, methods for determining the success of bots were limited and made it difficult to gauge public service value despite this being a stated aim. There were no clear criteria for measuring value and success across the trials, yet these evaluations play an important role in informing the organisation’s understanding of the potential of bot technology. One technologist said she would ask “can we get something technically up and running?” and then “did we learn anything?” (I-7). Across the case studies there was no systematic gathering of comparable statistics, comprehensive application of analytics or common instruments for measuring success, meaning evaluation lacked common criteria for deciding whether a bot experiment was successful and whether it had public value. One man- ager said public service values were “built into everything we do” and come into the team’s thinking “right at the beginning” of any project (I-11). However, the only con- sistent criterion prioritised and applied to all trials was technical: did the bots perform the task they were supposed to and were there any bugs or problems? This is perhaps unsurprising given the remit in R&D concerns the technical side of innovation. The manager explained that for each bot he would ask, “does it increase engagement and reach, especially with underserved audiences?” (I-11) – a central tenet of public service provision, with underserved audiences including, amongst others, the young and black and minority ethnic communities. Methods for assessing how well some of the bots performed did include audience engagement in terms of “reach” and “clicks,” but

14 B. JONES AND R. JONES

metrics were not detailed enough to indicate whether the bots actually appealed to these target audiences or helped with their comprehension of the subject matter. Those involved in development acknowledged a need for improved measures and metrics to evaluate if the bots “add value” for public service news (I-11, I-7).

Members of editorial staff were asked for feedback – sometimes as short written reports or verbally. Developers and journalists put different weights on aspects used to evaluate success, with the former more focussed on whether/how efficiently the bots performed their assigned function. The latter – perhaps often taking for granted that the technology should work – focussed on the impact on their workflow and whether this impact, if cumbersome, was outweighed by the benefits they assumed the bot brought for audience reach and engagement. For example, the reporter working on the in-article Newsbot said it was “very difficult to measure success from such a small trial” but that, from an editorial perspective, “the primary measure for me would be longer engagement time… if they kept clicking” (I-1). He added: “It needs to pass two tests: Add something for the reader, which it did, and be realistic for the journalists, which with some reservations, it did” (I-1). It was important for him that bots in future should “free us up to do other things” that require journalistic skill by taking care of routine and repetitive tasks, a view found in other studies of automation in news (Van Dalen 2012). Developers, meanwhile, considered that “measuring the usefulness” was “not within the scope of the experiment” as they were looking to gauge “enthusiasm” for the bots (Document 1). It is important to note that both journalists and developers were making some assumptions about the success of the bots with the audience. Audience attitudes towards, and experiences of, the bots, were not formally or compre- hensively assessed, and nor were the implications for journalistic workflow.

The lack of formalised benchmarks for determining what success in a public service context would look like meant it was difficult to determine not only how each bot per- formed in comparison to the others but also what the wider public value of chatbots in journalism over other formats might be. This is an area where ambitions were stated but no rigorous assessment made of the extent to which those ambitions were met. For example, one project lead said bots should be about “relationship building” with the audience, getting them “engaged and developing trust over time” and giving a person “agency” to “follow their own paths” (I-10). Limited research at this stage of develop- ment can have wider implications as developers risk misunderstanding the impact of novel formats and technologies on editorial staff, audiences and public service journal- ism. Given the increasingly important role of technology in the delivery of PSM, prelim- inary research and evaluation should extend beyond technical matters, user experience and reductive measures of audience engagement such as clicks and time spent – stand- ard measures of engagement in the media industry (Baym 2013) – to include evalua- tions based on how much these technologies help achieve PSM goals.

Third Parties and Outsourcing

In addition to these findings, it is important to take account of the issues raised by the BBC’s engagement of third parties to develop three news bots – the BBC Politics Brexit bot, the 2017 General Election bot and NewsChatta4. Outsourced bot

DIGITAL JOURNALISM 15

development raises questions about how much control is ceded to third parties (Sørenson 2007), whether their technologies are open to scrutiny or “black-boxed” and whether they align with public service values. BBC strategists had begun assessing risks posed by bots and were aware of the constraints of working with third parties (I- 12), highlighting issues ranging from security compliance, and access to APIs, to “black boxing” (1-12). Assessments highlighted the BBC’s need to be able to access, audit and approve algorithms and access raw audience data collected by bot platforms (I- 12), which is particularly important for PSM because they must be accountable to reg- ulators and the public.

Outsourcing, by nature, requires PSM to cede control and oversight of the develop- ment of that technology to third parties. Editorial independence is a core public ser- vice value, but increasing reliance on commercial software “solutions” and third-party distribution platforms raises issues around dependence on, and independence from, commercial intermediaries and around transparency and accountability. Sørenson and Hutchinson see this as a potential “strategic vulnerability” (2018: 99) for PSM. Belair- Gagnon and Holton (2018) highlight for example how web analytics companies employed by news organisations have fostered disruptive, profit-oriented norms and values in newsrooms. In-house development in this study’s cases ensured the BBC retained technical and editorial control, but in instances where external companies are used it is vital that appropriate partnership strategies are put in place that ensure they adhere to BBC values and standards.

These case studies distributed pre-published and pre-approved BBC material, thus avoiding many editorial and ethical issues associated with delegation of content gen- eration to algorithmic and automated processes. The overarching challenge identified by interviewees was how to make journalism conversational using automated techni- ques while still maintaining trust and preserving editorial control – it was both about “relationship building” with the audience and “developing trust over time” (I-10) by balancing automation with curation (I-1). Bennet (2018, 118) comments on the import- ance of aligning algorithmic and editorial logics in PSM. As bots and other automated software shift the balance of responsibility for content from the editorial to the technological side, journalists must consider how these technologies are mediating content in order to exert meaningful control over output.

Chatbots: Moving the Conversation Forward

These bot experiments were solely trials, and only the in-article chatbot had been rolled out into BBC newsrooms. Conscious of risk to trust and brand, BBC News bot development has so far been measured and shows a desire to master basic capabil- ities before rushing to keep up with the industry’s drive towards AI in the market. The research identified an ambition to develop more intelligent, conversational and auto- mated journalism that does not appear to match the reality on the ground, partly due to a cautious approach taken to avoid undermining trust and partly to the difficulty of integrating new technology (particularly ML) into legacy systems. The BBC’s circum- spect approach is in line with other PSM and contrasts with more highly automated, algorithmic and ML-driven experiments amongst commercial media and bot platforms.

16 B. JONES AND R. JONES

Taken individually, the bots might not have a significant impact. However, they form part of the organisation’s wider moves towards algorithmically personalised news and individually-oriented modes of addressing audiences, and further consideration of potential cumulative impacts and challenges to public service journalism is necessary to provide a counterpoint to the immediately perceptible opportunities. If conversa- tional journalism becomes a dominant paradigm in news provision without due critical assessment, what appear to be isolated and negligible transformations may end up shaping public service news in unforeseen ways. This research indicates that in the process of experimentation, the BBC is developing new infrastructure, skills and tech- nologies for bots and other CUIs and is shaping future formats for engaging with news. New job roles have been created fusing editorial and technical skills, and, importantly, rules, guidance and policy are being developed to standardise currently disparate approaches to CUIs and the use of third-party companies – an important area which PSM must think about critically and which is ripe for future research.

Following these trials, the BBC has begun investing in tools that can scale up use of bots, for example by creating a “BotBuilder” that “automatically generates a database of questions and answers from an inputted article URL” or from bespoke text to quickly make a bot and simplify the process for journalists (BBC News Labs 2018). It is also looking to enhance the technical capacity of CUIs, for example by building a chatbot with which a human can “converse” using free text. Meanwhile, the recent turn towards voice interaction – exemplified by Microsoft’s Cortana, Apple’s Siri and HomePod, Amazon’s Alexa and Echo, and Google’s Assistant and Home – has seen BBC R&D expand into this area (Barot 2017). It is exploring the potential of combining voice technology (using ML applications such as NLP), with more sophisticated audience data analytics and recommender algorithms5. Exploring “meaning-making” in these news contexts of human-machine communication will be crucial to understanding how technology can be conceptualised as more than a channel or medium, entering into the role of a communicator (Guzman 2018; Lewis, Guzman and Schmidt 2019).

Ensuring scrutiny of these processes and putting methods of accountability in place will be difficult but a vital part of applying public service values to future BBC journal- ism and, ultimately, of maintaining trust with the audience. More ML in content pro- duction and distribution creates opportunities for broadcasters but also introduces further challenges, some of which the BBC is already familiar with, such as transpar- ency, bias, accountability, and trust. However, there are distinctly new dimensions to these challenges as they become increasingly interwoven into the design and devel- opment of new data-driven technologies. For example, in order to apply ML to con- versational agents in ways that do not undermine trust in PSM news (Waddell 2017), there will need to be: transparency around training data sets – making them available for scrutiny; explainability regarding the role of algorithms in editorial output – mak- ing algorithmic aspects visible and providing explanations that justify their use; accountability built into new processes and new forms of distributed responsibility – identifying who is responsible when things go wrong (Weeks 2014; Montal and Reich 2017); and conversational agents that do not unintentionally discriminate – ensuring personalisation treats people fairly. Reliance on commercial ML “solutions” and third- party platforms raises important questions around degrees of dependence on, or

DIGITAL JOURNALISM 17

independence from, commercial interests and around degrees of commercial interfer- ence. The BBC must develop in ways that ensure its values are embedded and pre- served in the application of these technologies as part of its public service. The key question for the BBC is not simply a technical one; it is about developing applications of conversational technologies and machine learning that enshrine and bolster the public purpose.

These developments, alongside others occurring across the BBC, are changing the way journalists, managers and developers think about what is possible and may be altering the expectations that audiences have of their engagement with the BBC and news more generally. Further research is needed to analyse bots in contexts where they are beyond the trial stage and embedded in news production in order to gain insight into how they are shaping, and shaped by, journalists’ routines, professional identity, and perception of bots and automated technology. This should be comple- mented by audience research probing the relationship with, and impact on, news consumption.

Conclusion

This study sheds light on how the BBC has experimented with bots, and particularly chatbots, primarily for news distribution rather than for meaningful interaction with audiences. Recognising that technologies, as socio-material artefacts, are enacted in material-discursive practices (Bucher 2018) through a process of mutual shaping in context, our analysis illuminated not only the functions of BBC news bots but also the ways in which key actors conceived of, documented, and evaluated them. It indicated consensus over the meaning of bots was yet to stabilise and illustrated how bots were being mobilised as a vehicle to test out novel forms of individualised, personal- ised and two-way interaction, reflecting the popular conversational tone prevalent on social media. It suggested a shared organisational narrative – voiced by both technolo- gists and journalists – was being marshalled around the value of bots in PSM for improving reach and engagement, particularly with underserved audiences including young people, but also suggested that limited evaluation did not adequately assess this. It highlighted concern that increased automation and the increased algorithmic complexity of bots should not undermine editorial control and oversight. These bots are concrete examples of how the BBC is translating evolving notions of journalism as a more accessible and less abstract public good into the technologies it is developing. In future, bots – particularly chatbots and CUIs – seem likely to play a part in revising the relationship between public service broadcasters such as the BBC and their audi- ence. BBC News bots at the time of this research were not AI- or ML-driven and their limited technical capabilities and reliance on pre-written BBC sourced text did not pre- dispose them to some of the ethical issues foregrounded by more algorithmically gen- erated content. However, this study captures an important moment in the changing face of BBC News, which plans to build from its experiences developing these bots to experiment with AI for conversational journalism. We find evidence of efforts to stabil- ise this form of technology as the BBC begins to rein in the ad hoc elements of devel- opment and pin down a strategy for future work on conversational agents by

18 B. JONES AND R. JONES

formalising development processes, relationships with platforms and, importantly, with partners. Decisions about whether to develop in-house or outsource are likely to be of increasing importance and pose one of the most pressing challenges in ensuring public service values are preserved, warranting further research. Bots – specifically chatbots – at the BBC were seen as an opportunity to reach and engage underserved audiences by making the news more accessible to young people and those who are typically turned off by the formality of journalism. But their contribution to achieving PSM goals is unclear due to limited organisational research into audience responses. This is fertile ground for enquiry, particularly as the levels of intelligence and algorithmic complexity of bots increase.

Notes

1. A distinction can be made between AI and ML, the latter being any technology that allows computers to learn directly from examples and experience in the form of data, and the former being an umbrella term for the science of making machines “intelligent.” There is no agreed definition of AI, but most stress that AI systems are those that automate aspects of human intelligence. ML is a sub-field or narrow application of AI. The BBC is currently thinking about AI largely in terms of ML.

2. Author one is a BBC Broadcast Journalist for BBC News Online and author two is a Senior Researcher in BBC Research & Development.

3. Additionally, there is ongoing work on a “breaking news bot” but this has not yet advanced to trial stage.

4. In 2017, BBC Politics commissioned UK-based bot technology agency The Bot Platform to produce a Brexit Facebook Messenger bot, which used push notifications (e.g., for constituency results), a quiz and news updates. It then hired the same company for the 2017 UK General Election bot, which provided the latest news on the campaign, and information about the parties and their policies. Also in 2017, it launched the NewsChatta chatbot on WeChat for a Nigerian audience, developed again by a third party – Nigerian company Codulab.

5. See https://www.bbc.co.uk/rd/projects/talking-with-machines and http://bbcnewslabs.co.uk/ projects/voice-user-interfaces/.

Acknowledgements

We would like to acknowledge the help of the BBC, where both authors are employed and in particular the team at BBC News Labs.

Disclosure statement

In accordance with Taylor & Francis policy and our ethical obligation as researchers, we are reporting that we are both employed by the BBC, which may be affected by the research reported in the enclosed paper. I have disclosed those interests fully to Taylor & Francis, and I have in place an approved plan for managing any potential conflicts arising from this employment.

Funding

No grants were used to support this research.

DIGITAL JOURNALISM 19

ORCID

Bronwyn Jones http://orcid.org/0000-0003-2482-5181 Rhianne Jones http://orcid.org/0000-0002-8749-9953

References

Anderson, Chris W. 2011. “Deliberative, Agonistic, and Algorithmic Audiences: Journalism’s Vision of its Public in an Age of Audience Transparency.” International Journal of Communication 5: 529–547.

Anderson, Chris W. 2012. “Towards a Sociology of Computational and Algorithmic Journalism.” New Media and Society 15 (7): 1005–1021.

Barot, Trushar. 2015. “The Botification of News”. Nieman Lab. December. http://www.niemanlab. org/2015/12/the-botification-of-news/

Barot, Trushar. 2017. “The Future of News is Humans Talking to Machines”. Nieman Lab, September. http://www.niemanlab.org/2017/09/the-future-of-news-is-humans-talking-to-machines/.

Baym, Nancy K. 2013. “Data Not Seen, the Uses and Shortcomings of Social Media Metrics.” First Monday 18 (10).

Baym, Nancy K., and Boyd, Danah. 2012. “Socially Mediated Publicness: An Introduction.” Journal of Broadcasting & Electronic Media 56 (3): 320–329.

BBC. 2017. Written Evidence (AIC0204). Artificial Intelligence Committee Submission. http://data. parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/artificialintelligence- committee/artificial-intelligence/written/70493.pdf Accessed 18 June 2018.

BBC News Labs. 2018. “Scripting chatbots is hard. Here’s how we made it easier for BBC journalists: In our toolkit: BBC News BotBuilder”. Medium. June. https://medium.com/bbc-news-labs/ bbc-botbuilder-ba8e09b6a2e9.

Belair-Gagnon, Valerie, and Holton, Avery E. 2018. “Boundary Work, Interloper Media, and Analytics In Newsrooms.” Digital Journalism, 6 (4): 492–508.

Bennet, James. 2018. “Public Service Algorithms.” In A Future for Public Service Television, edited by Des Freedman and Vana Goblot. London: Goldsmiths Press.

Bucher, Taina. 2018. If… Then: Algorithmic Power and Politics. New York, N.Y: Oxford University Press.

Carlson, Matt. 2015. “The Robotic Reporter: Automated Journalism and the Redefinition of Labor, Compositional Forms, and Journalistic Authority.” Digital Journalism 3 (3): 416–431.

Carlson, Matt. 2018. “Confronting Measurable Journalism”. Digital Journalism, 6 (4): 406–441. Caswell, David, and D€orr, Konstantin. 2017. “Automated Journalism 2.0: Event-Driven Narratives:

From Simple Descriptions to Real Stories.” Journalism Practice 12 (4): 477–496. Coddington, Mark. 2015. “Clarifying Journalism’s Quantitative Turn: A Typology for Evaluating

Data Journalism, Computational Journalism, and Computer-Assisted Reporting.” Digital Journalism 3 (3): 331–348.

DeLanda, Manuel. 2006. A New Philosophy of Society: Assemblage Theory and Social Complexity. London: Continuum.

Denzin, Norman K. 1970. The Research Act: A Theoretical Introduction to Sociological Methods. New York: Aldine.

D€orr, Konstantin. 2015. “Mapping the Field of Algorithmic Journalism.” Digital Journalism. 4 (6): 700–722.

Flew, Terry, Spurgeon, Christina, Daniel, Anna, and Swift, Adam. 2012. “The Promise of Computational Journalism.” Journalism Practice 6 (2): 157–171.

Gehl, Robert W., and Bakardjieva, Maria (Eds.). 2016. Socialbots and Their Friends: Digital Media and the Automation of Sociality. New York, N.Y: First Edition. Routledge.

Gillespie, Tarleton, Boczkowski, Pablo and Foot, Kirsten. 2014. Media Technologies: Essays on Communication, Materiality, and Society. Cambridge, MA: The MIT Press.

Graves, Lucas. 2018. “Boundaries Not Drawn.” Journalism Studies 19 (5): 613–631.

20 B. JONES AND R. JONES

Gorwa, Robert, and Guilbeault, Douglas. 2018. “Unpacking the Social Media Bot: A Typology to Guide Research and Policy.” Policy & Internet. doi:10.1002/poi3.184.

Guzman, Andrea. 2018. “What is Human-Machine Communication, Anyway?” In Human-Machine Communication: Rethinking Communication, Technology, and Ourselves, edited by Andrea Guzman. Chapter: Introduction, 1–28. New York, NY: Peter Lang.

Hall, Tony. 2017. “Tony Hall’s speech at the launch of the Annual Plan for 2017/18.” BBC website. July. http://www.bbc.co.uk/mediacentre/speeches/2017/tony-hall-annual-plan#heading-a-personalised- uniquely-tailored-bbc

He, Lei. 2016. “Behind the BBC’s Messenger News Bot.” BBC News Labs Blog. November http:// bbcnewslabs.co.uk/2016/11/16/mundo-for-messenger/.

Helberger, Natali. 2015. “Merely Facilitating or Actively Stimulating Diverse Media Choices? Public Service Media at the Crossroad.” International Journal of Communication 9: 1324–1340.

Holton, Avery and Belair-Gagnon, Valerie. 2018. “Strangers to the Game? Interlopers, Intralopers, and Shifting News Production”. Media and Communication 6 (4): 70–78.

Jones, Rupert. 2018. “NatWest Bank tests Cora, an AI bot that will answer customer questions.” Guardian Online. https://www.theguardian.com/money/2018/feb/21/natwest-bank-tests-cora- an-ai-bot-that-will-answer-customer-questions.

Keohane, Joe. 2017. “What news-writing bots mean for the future of journalism” Wired. February. https://www.wired.com/2017/02/robots-wrote-this-story/.

Lewis, Seth C., Guzman, Andrea and Schmidt, Thomas. 2019. Automation, Journalism, and Human–Machine Communication: Rethinking Roles and Relationships of Humans and Machines in News. Digital Journalism. 1–19. doi:10.1080/21670811.2019.1577147.

Lewis, Seth C., and Usher, Nikki. 2013. “Open Source and Journalism: Toward New Frameworks for Imagining News Innovation.” Media, Culture & Society, 35 (5): 602–619.

Lewis, Seth C., and Usher, Nikki. 2014. “Code, Collaboration, and the Future of Journalism: A Case Study of the Hacks/Hackers Global Network”. Digital Journalism. 2 (3): 383–393. doi: 10.1080/21670811.2014.895504

Lewis, Seth C., and Usher, Nikki. 2016. “Trading Zones, Boundary Objects, and the Pursuit of News Innovation: A Case Study of Journalists and Programmers.” Convergence: The International Journal of Research into New Media Technologies 22 (5): 543–560.

Lewis, Seth C., and Westlund, Oscar. 2015. “Actors, Actants, Audiences, and Activities in Cross- Media News Work”. Digital Journalism 3 (1): 19–37.

Lokot, Tetyana and Diakopoulos, Nick. 2015. “News Bots: Automating News and Information Dissemination on Twitter.” Digital Journalism 4 (6), 682–699. doi:10.1080/21670811.2015. 1081822.

Lotz, Amanda D. 2018. “Inventing Public Service Media.” In A Future for Public Service Television, edited by Des Freedman and Vana Goblot. London: Goldsmiths Press.

McKinney, Sydney. 2018. “Tech & Check Alerts aim to ease the workload of fact-checkers.” Duke Reporters Lab. April. https://reporterslab.org/tech-and-check-alerts-automated-fact-checking/

Montal, Tal and Reich, Zvi. 2017. “I, Robot. You, Journalist. Who is the Author?” Digital Journalism 5 (7): 829–884.

Napoli, Philip M. 2014. “Automated Media: An Institutional Theory Perspective on Algorithmic Media Production and Consumption.” Communication Theory 24 (3): 340–360.

Neff, Gina and Nagy, Peter. 2016. “Talking to Bots: Symbiotic Agency and the Case of Tay.” International Journal of Communication 10: 4915–4931.

Newman, Nic. 2018. “Digital News project 2018. Journalism, Media, and Technology Trends and Predictions 2018.” Reuters Institute for the Study of Journalism. https://reutersinstitute.politics. ox.ac.uk/sites/default/files/2018-01/RISJ%20Trends%20and%20Predictions%202018%20NN.pdf

Nielsen, Rasmus Kleis, 2012. “How Newspapers Began to Blog: Recognizing the role of technolo- gists in old media organizations’ development of new media technologies”. Information, Communication, and Society 15 (6): 959–978.

O’Donnell, Fionnt�an. 2017. “What We Talk About When We Talk About Fair AI.” Medium, https:// medium.com/bbc-news-labs/what-we-talk-about-when-we-talk-about-fair-ai-8c72204f0798

DIGITAL JOURNALISM 21

Phelps, Andrew. 2017. “This is How The New York Times is Using Bots to Create More One-To- One Experiences with Readers.” Nieman Lab. http://www.niemanlab.org/2017/04/this-is-how- the-new-york-times-is-using-bots-to-create-more-one-to-one-experiences-with-readers/

Picard, Robert. 2010. “The Future of the News Industry.” In Media and Society, edited by J. Curran, 365–379. London: Bloomsbury Academic.

Ponsford, Dominic. 2017. “First Robot-Written Stories from Press Association Make it into Print in ‘World-First’ for Journalism Industry.” Press Gazette http://www.pressgazette.co.uk/first-robot- written-stories-from-press-association-make-it-into-print-in-world-first-for-journalism-industry/

Powers, Matthew. 2012. “In Forms That are Familiar and Yet-to-be Invented’: American Journalism and the Discourse of Technologically Specific Work.” Journal of Communication Inquiry 36 (1): 24–43.

Scannell, Paddy. 2005. “The Meaning of Broadcasting in the Digital Era.” In Cultural Dilemmas in Public Service Broadcasting, edited by Gregory Ferrell Lowe and Per Jauert, 129–142. G€oteborg: Nordicom.

Sehl, Annika, Cornia, Alessio, and Nielsen, Rasmus Kleis. 2016. “Public Service News and Digital Media. Digital News Project 2016.” Reuters institute for the Study of Journalism.

Sørenson, Jannick Kirk. 2007. “The Paradox of Personalisation.” PhD Thesis. Available at https:// www.researchgate.net/publication/295825824_The_Paradox_of_Personalisation_Public_Service_ Broadcasters%27_Approaches_to_Media_Personalisation_Technologies . Accessed 20.5.2018.

Sørenson, Jannick Kirk, and Hutchinson, Jonathon. 2018. “Algorithms and Public Service Media. In Public Service Media in the Networked Society” RIPE 2017, pp. 91–106.

Thurman, Neil, D€orr, Konstantin, and Kunert, Jessica. 2017. “When Reporters Get Hands-On with Robo-Writing: Professionals Consider Automated Journalism’s Capabilities and Consequences.” Digital Journalism 5 (10): 1240–1259.

Van Dalen, Arjen. 2012. “The Algorithms Behind the Headlines: How Machine-Written News Redefines the Core Skills of Human Journalists”. Journalism Practice 6 (5–6): 648–658.

Van Dyke, Jose and Poell, Thomas. 2015. “Making Public Television Social? Public Service Broadcasting and the Challenges of Social Media.” Television & New Media 16 (2): 148–164.

Van Es, Karen. 2017. “An Impending Crisis of Imagination Data-Driven Personalization in Public Service Broadcasters.” Media@LSE working papers. 43: 1–18. https://dspace.library.uu.nl/handle/ 1874/358206

Waddell, T. Franklin. 2017. “A Robot Wrote This? How Perceived Machine Authorship Affects News Credibility.” Digital Journalism 6 (2): 236–255.

Wang, Shan. 2017. “From Election Results to Sports, The Hindustan Times is Trying Out Twitter Bots for Live Coverage.” Nieman Lab. http://www.niemanlab.org/2017/06/from- election-results-to-sports-the-hindustan-times-is-trying-out-twitter-bots-for-live-coverage/

Weeks, Lin. 2014. “Media Law and Copyright Implications of Automated Journalism.” NYU Journal of Intellectual Property and Entertainment Law 4: 67–94. https://jipel.law.nyu.edu/vol-4-no-1-3-weeks/

Willnat, Lars, Weaver, David H. and Choi, Jihyang. 2013. “The Global Journalist in the Twenty-First Century: A Cross-National Study of Journalistic Competencies.” Journalism Practice 7 (2): 163–183.

Woolley, Samuel and Howard, Philip N. 2017. “Computational Propaganda Worldwide: Executive Summary”. Edited by Samuel Woolley and Philip N. Howard, eds. Working Paper No. 11. Project on Computational Propaganda. Oxford Internet Institute. University of Oxford, UK.

You, Jia. 2015. “ONA15: How News Organizations Build Simple Bots to Help Report the News. Northwestern University Knight Lab.” https://knightlab.northwestern.edu/2015/10/09/ona15- how-news-organizations-build-simple-bots-to-help-report-the-news/

Young, Mary-Lynn, and Hermida, Alfred. 2014. “From Mr and Mrs Outlier to Central Tendencies: Computational Journalism and Crime Reporting at the Los Angeles Times.” Digital Journalism 3 (3): 381–397.

Zuiderveen Borgesius, Frederik J., Trilling, Damian, M€oller, Judith, Bal�azs, Bod�o, de Vreese, Claes H., Helberger, Natali. 2016. “Should We Worry About Filter Bubbles.” Internet Policy Review 5 (1): 1–16.

22 B. JONES AND R. JONES