Application 1 – Analysis and Synthesis of Prior Research

S P E C I A L I S S U E - R E ’ 0 7 B E S T P A P E R S

Exploring how to use scenarios to discover requirements

Norbert Seyff Æ Neil Maiden Æ Kristine Karlsen Æ James Lockerbie Æ Paul Grünbacher Æ Florian Graf Æ Cornelius Ncube

Received: 5 September 2008 / Accepted: 29 January 2009 / Published online: 24 February 2009

� Springer-Verlag London Limited 2009

Abstract This paper investigates the effectiveness of dif-

ferent uses of scenarios on requirements discovery using

results from requirements processes in two projects. The first

specified requirements on a new aircraft management system

at a regional UK airport to reduce its environmental impact.

The second specified new work-based learning tools to be

adopted by a consortium of organizations. In both projects

scenarios were walked through both in facilitated workshops

and in the stakeholders’ workplaces using different forms of

a scenario tool. In the second project, scenarios were also

walked through with a software prototype and creativity

prompts. Results revealed both qualitative and quantitative

differences in discovered requirements that have potential

implications for models of scenario-based requirements

discovery and the design of scenario tools.

1 Different scenario uses

Scenarios are sequences of events with a narrative structure

[2]. They are simple, human things [2], and walking through

them is one of the more effective means by which stake-

holders discover requirements. Studies have reported

stakeholders walking through different types of scenarios,

from simple stories to surface new requirements [8] to sys-

tem simulations to discover emergent system properties [1,

10]. However, despite reported successes, we still lack data

with which to determine what are the more effective uses of

scenarios for discovering requirements. This paper reports

results from two scenario-driven processes to discover

requirements for an air traffic management system called

VANTAGE (Validation of Network-Centric, Technology

Rich ATM System Guided by the Need for Environmental

Governance) and a system to support work-integrated

learning in organizations called APOSDLE (Advanced

Process-Oriented Self-Directed Learning Environment).

ART-SCENE is a software environment for discovering

and documenting stakeholder requirements [17] by walk-

ing through scenarios that are automatically generated from

use case specifications. We run what we call scenario

workshop walkthroughs for same-time same-place discov-

ery of requirements using the desktop version of ART-

SCENE [17]. A workshop is a structured meeting that is

ran by a trained facilitator to ensure effective stakeholder

input to the meeting. The facilitator drives the walkthrough

process whilst a scribe documents requirements and sce-

nario changes in ART-SCENE. Similar uses of scenarios

are reported in [9, 26]. Stakeholders walk through one

ART-SCENE-generated scenario displayed to them on a

large screen. In the VANTAGE project we ran four sce-

nario workshop walkthroughs to discover requirements.

Walking through software prototypes and scenarios

together has been shown to improve requirements com-

pleteness [28]. In the APOSDLE project a first prototype of

the work-integrated learning system had already been

developed. Therefore, we ran eight scenario workshop

walkthroughs that walked through ART-SCENE scenarios

and the APOSDLE prototype together. Each scenario

N. Maiden (&) � K. Karlsen � J. Lockerbie Centre for HCI Design, City University London,

London EC1V 0HB, UK

e-mail: [email protected]

N. Seyff � P. Grünbacher � F. Graf Systems Engineering and Automation,

Johannes Kepler University, 4040 Linz, Austria

C. Ncube

Software Systems Research Centre, Bournemouth University,

Fern Barrow, Talbot Campus, Poole, Dorset BH12 5BB, UK

123

Requirements Eng (2009) 14:91–111

DOI 10.1007/s00766-009-0077-9

workshop walkthrough was supplemented by results from

an earlier creativity workshop held in the project [15] that

provided additional vision, requirements and design fea-

tures for future versions of the APOSDLE system.

However bringing stakeholders together in workshops

can be difficult and time-consuming, whilst removing them

from their workplace can miss important contextual trig-

gers for requirements. One alternative is to walk through

scenarios in the workplace using mobile technologies.

Previously we developed a version of ART-SCENE called

the Mobile Scenario Presenter (MSP) to run on a Personal

Digital Assistant (PDA). Evaluations of the MSP [19, 20]

revealed that it could be used to discover requirements in

the workplace; however analysts found it difficult to doc-

ument requirements using the stylus when moving and/or

communicating with the observed stakeholders. Therefore

we also walked through 1 ART-SCENE scenario in the

VANTAGE project and four different ART-SCENE sce-

narios in the APOSDLE project in the workplace to

provide empirical data about their effectiveness. We call

these walkthroughs scenario workplace walkthroughs.

We used data from the scenario workshop walkthroughs

and scenario workplace walkthroughs in the VANTAGE

and APOSDLE projects to answer three research questions:

Q1 Can a scenario workshop walkthrough, supported

with software prototypes and design features, trigger a

larger number of requirements than the walkthrough of

the scenario on its own?

Q2 Can a scenario workplace walkthrough trigger

requirements that might not be discovered with a

scenario workshop walkthrough?

Q3 Can a scenario workplace walkthrough trigger a

larger number of requirements than an equivalent

scenario workshop walkthrough?

The first research question Q1 explored whether sup-

plementing scenarios with design knowledge in the

software prototypes and creativity prompts would increase

the number of requirements generated. We distinguished

between Q2 and Q3 to investigate whether scenario

workplace walkthroughs generated different requirements

from scenario workshop walkthroughs.

In the remainder of the paper, Sect. 2 reports the dif-

ferent uses of scenarios that were applied in VANTAGE

and APOSDLE. Section 3 reports a model of scenario-

based discovery that informs the three research questions.

Sections 4 and 5 report results from the scenario walk-

throughs to answer the research questions. Section 6 uses

the results to answer the three research questions and

explore their validity. Sections 7 and 8 present lessons

learned for running future scenario walkthroughs and

report related work. Section 9 outlines future research to

improve scenario use in requirements processes.

2 ART-SCENE scenario walkthroughs

The VANTAGE and APOSDLE scenarios were generated

and walked through using the ART-SCENE environment.

2.1 ART-SCENE scenarios

The big idea that underpins ART-SCENE scenario walk-

throughs is very simple—that people are better at

identifying errors of commission rather than omission [4].

From this general trend in human cognition for recall to be

weaker than recognition, ART-SCENE scenarios in the

ART-SCENE software environment offer stakeholders

recognition cues in the form of automatically generated

alternative courses. If the alternative course is relevant to

the system being specified but not yet handled in the

specification, then a potential omission has been identified,

and ART-SCENE guides the analysts to specify and doc-

ument the relevant requirements.

ART-SCENE automatically generates scenarios in two

steps [17]. In the first it generates different normal course

scenarios from ordering rules in the use case specification.

Each different possible ordering of normal course events is a

different scenario. In the second the algorithm generates

candidate alternative courses, which are expressed as ‘what-

if’ questions for each normal course event, by querying a

database that implements a simple model of over 40 abnor-

mal behaviours and states in socio-technical systems

independent of the domain of interest. Some class hierarchies

were derived from definitions of scenario concepts such as

events and actions. Others were derived from error taxono-

mies in the cognitive science, human-computer interaction

and safety-critical disciplines [25]. ART-SCENE allows

specialization of these classes to selected domains such as air

traffic management and learning, as reported in [21].

ART-SCENE provides scenarios to stakeholders to

recognize events and discover requirements in two ways—

in scenario workshop walkthroughs and scenario work-

place walkthroughs. Each is described in turn.

2.2 Scenario workshop walkthroughs

In scenario workshop walkthroughs stakeholders interact

with one ART-SCENE component called the Scenario

Presenter. Figure 1 depicts one scenario generated for

VANTAGE and presented using the Scenario Presenter. A

scenario workshop walkthrough typically lasts half a day

[17]. A facilitator guides the stakeholders to recognise

which scenario normal and alternative course events—

what-if capabilities—the new system must handle. The

facilitator then uses simple heuristics to discover one or

more requirements that, if satisfied, will enable the system

to avoid, respond to or mitigate the effects of the event.

92 Requirements Eng (2009) 14:91–111

123

The scribe documents all requirements in ART-SCENE

and links them to the scenario events that triggered them.

In a scenario workshop walkthrough the facilitator

walks the stakeholders through an ART-SCENE scenario,

but these scenarios can be supplemented with other arte-

facts. In APOSDLE we ran scenario walkthrough

workshops, in which we walked through related design

features of an existing work-integrated software prototype

using an electronic whiteboard that stakeholders could

manipulate directly by touch to sketch and document

changes to supplement the requirements documented in

ART-SCENE. We also searched for and presented results

from an earlier creativity workshop [15] on a third display

to reuse visions, requirements and design ideas that had

been generated by stakeholders earlier in the requirements

process. As a result, at any time during each workshop,

stakeholders were interacting with the ART-SCENE sce-

nario, existing software prototype and documented ideas

from earlier activities.

However, one limitation of scenario workshop walk-

throughs is that stakeholders need to take time out of their

workplace to participate. As well as restricting access to

stakeholders who might not have the time to participate,

these sessions take place out of the workplace, thus

potentially diminishing the effectiveness in the require-

ments discovery process [30].

2.3 Scenario workplace walkthroughs

The MSP [19] is a PDA-based ASP.NET web application

that uses a mobile browser and wireless access to connect

to server-side ART-SCENE scenario and requirements

databases. The tool is optimized for Microsoft’s Pocket

Internet Explorer included with Microsoft’s Pocket PC OS.

The MSP allows its user to discover and document

requirements systematically in the workplace using struc-

tured scenarios generated by ART-SCENE. The MSP user

walks through scenarios of future system behaviour and

observes current system behaviour at the same time. What-

if capabilities—generated candidate alternative courses for

each event—enable the user to follow-up and ask questions

about abnormal and unusual behaviour in different work-

places, thus leading to more complete requirements

discovery. Figure 2 shows some normal and alternative

Fig. 1 One VANTAGE scenario (VA2 Approach/Arrival Sequence Control) in the desktop version of ART-SCENE used during a scenario workshop walkthrough

Requirements Eng (2009) 14:91–111 93

123

course events of a second VANTAGE scenario, VA4On-

stand Operations, that we walked through using the MSP.

One important enhancement to the MSP used in VAN-

TAGE was audio recording of requirements. Earlier

versions captured new requirements in typed form using the

PDA stylus [9]. However, this was less successful than

expected due to the effort needed to type requirements.

Therefore, the reported scenario workplace walkthroughs

took place with audio recording of spoken requirements. To

deliver this capability the MSP implemented a new plug-in

solution on top of the full-screen browser application. The

solution was integrated into the menu of the full-screen

browser. The capability could be started using one touch

with the stylus once the plug-in solution was enacted. Audio

files of generated requirements were stored using common

file formats so that analysts could continue their work on the

desktop ART-SCENE after synchronization. All created

multimedia files were linked to the underlying MSP data-

base using special IDs included in the filename.

Another change undertaken in VANTAGE and APOS-

DLE was the process of the scenario workplace

walkthrough. In previous applications we replaced work-

shops with a two-stage process—one-on-one observations

and fact gathering by a single analyst in the workplace

followed by project-wide interpretation sessions using the

desktop Scenario Presenter. However, this procedure was

problematic. A single analyst was not able to observe the

workplace, navigate the scenario, communicate with

stakeholders and document requirements using a stylus

because earlier audio-recording features were too cum-

bersome to use [19]. Therefore, in VANTAGE and

APOSDLE, we introduced the two roles of facilitator and

scribe. Whilst the scribe used the MSP to navigate between

scenario events and to document requirements and com-

ments, the facilitator observed the workplace, asked

questions about it, and filtered the raw data to generate

first-cut requirement statements, based on recognition cues

provided by the MSP.

3 A model of scenario-based requirements discovery

We investigated the scenario-driven requirements pro-

cesses in VANTAGE and APOSDLE to answer the three

research questions reported in Sect. 1 about the effective-

ness of different scenario walkthrough types. The questions

were grounded in a logical task model that describes

essential cognitive tasks that stakeholders undertake during

a scenario walkthrough to specify a future system using

tools such as the Scenario Presenter.

The model describes scenario-based requirements dis-

covery as an iteration of tasks. Stakeholders read scenario

event descriptions and recognize one or more as possible

and to be handled by the system. Recognizing whether an

event is relevant is an essential pre-requisite to discovering

requirements. Because stakeholders are better at recog-

nizing incorrect events rather than recalling missing ones,

the model predicts that stakeholders will discover more

requirements using scenario events that are presented to

them to recognize as relevant than they will from unaided

recall of such events. For each event recognized as possi-

ble, stakeholders generate one or more requirements.

Fig. 2 The VANTAGE scenario VA4 On-stand Operations in the MSP version of ART-SCENE used in a

scenario workplace walkthrough

94 Requirements Eng (2009) 14:91–111

123

We developed and validated the model for scenario

workshop walkthroughs with tools such as the ART-

SCENE Scenario Presenter. However, extending our sce-

narios, such as with prototypes, creativity prompts, and

walking through scenarios in the workplace challenges the

assumptions behind the model. For example, software

prototypes and creativity cues explicitly describe design

knowledge with which to infer requirements about the new

system, which might generate more requirements than

simply recognizing unhandled events in a scenario. On the

other hand, design decisions embedded in the software

prototype might lead to more requirements that encapsulate

design decisions and assumptions. Therefore we explored

the effect of recognition cues from scenario events, soft-

ware prototypes and creativity prompts to answer Q1 using

data from the APOSDLE scenario walkthroughs. Likewise,

walking through scenarios in the workplace might expose

analysts and stakeholders to more event recognition cues.

However the timing of these events cannot be controlled,

thus making it more difficult for analysts to capture and

specify new requirements. We also explored the effect of

combined recognition cues from scenarios and the work-

place and audio requirements recording techniques to

answer Q2 and Q3 with data from the scenario workplace

walkthroughs.

The types of dependent variable data that were used to

answer the three research questions are listed in Table 1.

The generated requirements data was drawn directly from

ART-SCENE’s scenario and requirements databases, and

scenario walkthrough data was taken from observational

notes recorded by the facilitator and scribe during the

scenario walkthroughs.

We did not investigate stakeholder perceptions about the

walkthroughs and the scenarios due to difficulties capturing

such data in the VANTAGE and APOSDLE projects.

Stakeholders such as pilots and business consultants were

simply not available to be debriefed at the end of sessions,

and the availability of other stakeholders was restricted

during both projects.

The next two sections report the scenario walkthroughs

and results for the VANTAGE and APOSDLE projects,

respectively.

4 Walking through scenarios in VANTAGE

We walked through scenarios to discover requirements for

VANTAGE Phase-1, a system to reduce the environmental

impact of aircraft movements in and around airports. The

2-year project, funded by the UK’s Department of Trade

and Industry, integrated new technologies into the opera-

tions of regional airports in the United Kingdom to reduce

their environmental impact, measured as noise and gas

emissions. Partners who include Thales and Qinetiq intro-

duced new technologies at Belfast City Airport, BCA.

We walked stakeholders through scenarios as part of

RESCUE (Requirements Engineering with Scenarios in a

User-Centred Environment), a scenario-driven require-

ments process [18]. Prior to the walkthroughs the

requirements team had discovered requirements for the

new VANTAGE system using brainstorming sessions and

a creativity workshop, and generated requirements auto-

matically from i* models. A use case model specified 20

core use cases that specified how the future VANTAGE-

enhanced airport operations at BCA should behave during

landing, taxiing, on-stand operations and take-off, as well

as to support airport management such as producing daily

flight schedules. Requirements on the VANTAGE system

were associated with behaviour specified in the use cases.

However, we still needed VANTAGE stakeholders, who

included the BCA environmental manager, environmental

experts, technology specialists, and airline pilots and

operational staff, to discover complete requirements on

VANTAGE using projections of future operations at BCA.

This is where the scenario walkthroughs came in.

The scenario walkthroughs took place with real-world

project constraints such as time and availability of stake-

holders that could not be controlled in the walkthroughs

Table 1 Types of data used to answer three research questions

Generated requirements data Scenario walkthrough data

Total number of requirements generated in a

scenario walkthrough

Duration of the scenario walkthrough

Requirements description text Number of stakeholders presented in the

scenario walkthrough

Assigned requirement type

Scenario that triggered generation of the

requirement

Scenario event that triggered generation of

the requirement

Requirements Eng (2009) 14:91–111 95

123

without disrupting the project or invalidating its results.

Therefore, we adopted an action research approach, gen-

erating and analyzing data in context, taking care when

drawing conclusions from the results.

4.1 The scenario walkthrough schedule

We generated a scenario walkthrough schedule from the

VANTAGE use case model. VANTAGE stakeholders

prioritized the use cases for the potential of the specified

behaviour to minimize environmental impact. Not sur-

prisingly, given aircraft movements are the main source of

noise and gas emissions, the priority use cases specified

VANTAGE requirements on the arrival, turnaround and

departure of an aircraft from the airport.

Time and resource constraints meant that only five use

cases could be investigated. We set up the walkthrough

schedule in Table 2 and used the ART-SCENE scenario

generation algorithm to generate one scenario per use case.

We chose to run scenario workshop walkthroughs to

walk through scenarios that describe aircraft movement,

such as VA2 Approach and arrival sequence control and

VA11 Coordinate flight departures. Not only were scenario

workplace walkthroughs of such scenarios on the airport

difficult (not to say dangerous!), but scenario workshop

walkthroughs were tractable because representatives of

stakeholders such as airlines (pilots and dispatchers), air

traffic controllers and managers, the airport environment

manager, and solution experts from VANTAGE technical

partners were all available to attend them.

Figure 1 shows part of the ART-SCENE scenario for

VA2 Approach and arrival sequence control. Typical

events included dispatcher communicates with ramp staff

and BCA approach controller makes decisions about

landing approach. Each scenario workshop walkthrough

took place in a meeting room with one facilitator and one

scribe. Each was designed to run for 2–4 h, depending on

the number of events in the scenario.

The one scenario in which aircraft do not move was

more amenable to walk through in the workplace. The VA4

On-stand operationsscenario specifies all behaviour related

to the turnaround of an aircraft from when it parks to its

pushback. Part of the scenario is shown in Fig. 2. Typical

events included refuel the aircraft and disembark passen-

gers. The walk through of this scenario took place over 4 h

on a weekday. The analyst who facilitated the scenario

workshop walkthroughs also facilitated the scenario

workplace walkthrough. A different scribe operated the

MSP. The facilitator observed on-stand operations on dif-

ferent aircraft turned around by airlines and the local

service operator. He asked questions to airport and airline

staff being observed. All spoken requirements and com-

ments were recorded in the MSP.

Figure 3 shows the facilitator and scribe walking

through the scenario in the workplace. The left-hand side

depicts the scribe in the cockpit of an A321 aircraft whilst

the facilitator asked questions about ground system-aircraft

uploads, whilst the right-hand side shows the facilitator

asking questions of the operator during aircraft refueling.

Note the bad weather!

4.2 Generating the scenarios

In VANTAGE we generated one scenario for each of the

five prioritized use cases. Each scenario normal course

event sequence specified the expected event ordering dur-

ing aircraft approach, landing, taxiing, turnaround and

takeoff. For each normal course event in the five scenarios

the algorithm generated one or more candidate alternative

courses for each normal course event. Alternative courses

Table 2 VANTAGE scenario walkthrough schedule

Date Scenario Type of

walkthrough

11/09/2006 VA2: approach/arrival sequence control Workshop

11/09/2006 VA3: ground movement control arrivals Workshop

17/10/2006 VA1: ground movement control departure Workshop

17/10/2006 VA11: coordinate flight departures Workshop

29/11/2006 VA4: on-stand operations Workplace

Fig. 3 Two uses of the MSP when walking through the VA4 On-stand operations scenario in the workplace

96 Requirements Eng (2009) 14:91–111

123

were generated using the domain-independent version of

the ART-SCENE algorithm [17] not tailored to air traffic

control. VANTAGE scenario alternative courses expressed

abnormal behaviours and states that were domain-inde-

pendent, such as what if the dispatcher lacks the necessary

knowledge? and what if the aircraft exhibits some abnor-

mal behaviour?, rather than what if the aircraft uses the

wrong taxiway when arriving at the terminal? A generated

scenario specified on average 20 normal course events and

8 alternative course events per normal course event. The

shortest scenario was VA11 Coordinate flight schedules

with 8 normal course and 41 alternative course events. The

longest was VA4 On-stand operations with 35 normal

course and 271 alternative course events.

4.3 Scenario walkthrough results

All scenario walkthroughs took place as planned. Each

scenario workshop walkthrough lasted between 2 and 4 h.

The scenario workplace walkthrough lasted 4 h. Between 6

and 10 stakeholders attended each workshop depending on

availability—commercial airline pilots and air traffic con-

trollers were often unable to commit time to scenario

workshop walkthroughs, hence the workshop schedule was

a compromise between stakeholder availability and dead-

lines. Nonetheless all four included at least one

representative from BCA (e.g. an air traffic controller), an

airline (e.g. a BMI dispatcher and pilot), a solution tech-

nology provider (e.g. Thales and Raytheon), and an

environmental researcher from an academic partner in

VANTAGE. Many stakeholders participated in more than

one scenario workshop walkthrough.

Results from the scenario walkthroughs are reported in

Table 3. The five walkthroughs generated 147 require-

ments. All requirements were documented in ART-SCENE

using four attributes—the requirement description, ratio-

nale, type and source [23]. During scenario workshop

walkthroughs the scribe entered these fields using a desktop

computer keyboard, whilst during the scenario workplace

walkthrough the scribe selected the requirement type from

a pull-down menu, then the facilitator recorded the spoken

requirement description, rationale and source in the MSP.

After the walkthrough the facilitator and scribe transcribed

all spoken requirements, comments and scenario changes,

then the documented requirements were validated with

source stakeholders. In the scenario workshop walk-

throughs, the stakeholders were able to validate

requirements as the scribe entered them into the displayed

ART-SCENE requirements form.

Results reveal that walking through VA4 On-stand

operations in the workplace generated 59 requirements,

whilst the most requirements generated during one sce-

nario workshop walkthrough was 32, for VA2 Approach

and arrival sequence control. Each scenario workshop

walkthrough generated on average 22 requirements per

scenario. The average number of requirements generated

per scenario normal course event is also reported in

Table 3, but there was no discernible association between

scenario length and the number of requirements generated.

Alternative course events appeared to have little effect

on requirements discovery—only four requirements were

associated with alternative course events in the 5 scenarios.

One possible reason for this was the number of normal

course events in each scenario and limited time available in

each walkthrough. Each scenario workshop walkthrough

walked through the normal course events before the alter-

native course events. In most walkthroughs the time

available allowed stakeholders to walk through all normal

course events but not most alternative course ones.

Previous studies revealed that requirements documented

with the MSP and stylus were shorter than requirements

documented via keyboards [19]. In VANTAGE, require-

ments transcribed from the recordings of the spoken

requirements in the MSP were similar in length to

Table 3 The total number of requirements documented during each scenario, types of event for which requirements were documented, and average number of requirements generated per scenario normal course event, in the VANTAGE project

Scenario and walkthrough type Total number of

requirements

documented

Number of requirements

on use case and normal

course event behaviour

Number of requirements

on alternative

course behaviour

Average number of

requirements per

normal course event

VA2: approach and arrival sequence control

(workshop)

32 30 2 1.19

VA3: ground movement control arrivals

(workshop)

28 27 1 2.44

VA1: ground movement control departure

(workshop)

16 15 1 0.70

VA11: coordinate flight departures

(workshop)

12 12 0 1.5

VA4: on-stand operations (workplace) 59 59 0 1.55

Total 147 143 4

Requirements Eng (2009) 14:91–111 97

123

requirements typed by the scribe in the four scenario

workshop walkthroughs.

Table 4 reports the totals of requirement generated by

type. Walking through the scenario in the workplace led to

generation of more availability- and usability-type

requirements than during the scenario workshop

walkthroughs.

Occasionally the walkthroughs resulted in changes to

the scenarios themselves in response to stakeholder com-

ments and facilitator observations. The scenario workplace

walkthrough of VA4 On-stand operations resulted in 11

changes, including the addition of two observed new nor-

mal course events (e.g. the engineer enters the aircraft) and

nine observed alternative course events, for example what

if the stand is occupied? In contrast the four scenario

workshop walkthroughs resulted in just one change—an

event is undertaken by an air traffic support assistant rather

than a controller.

4.4 Qualitative requirements analysis

We investigated all 147 requirements to detect qualitative

differences between requirements associated with the two

types of scenario walkthrough.

4.4.1 Requirements subjects

The first characteristic was the subject of each requirement.

ART-SCENE mandated that all requirements were

expressed using shall statements with a common structure

[3] that leads to specification of properties of one or more

actors. These actors were the subjects of the requirements.

Results are reported in Table 5. Most requirements gen-

erated during the scenario workshop walkthroughs were

requirements on the VANTAGE software system. In con-

trast the VA4 On-stand operationsscenario workplace

walkthrough generated 20 requirements on the dispatcher

and 12 on the dispatch coordinator, both actors in on-stand

activities observed by the facilitator and scribe. And yet the

four scenario workshop walkthroughs only generated a

total of six requirements on the dispatcher, in spite of the

presence of dispatchers in each. This indicates that the type

of the scenario walkthrough, rather than dispatcher avail-

ability, influenced the generation of these 20 requirements.

The 12 requirements on the dispatch coordinator were

important in VANTAGE. In spite of earlier modeling of

airport operations with i*, the focus on dispatchers working

for airlines such as BMI rather than the airport services

operator led the project to overlook the dispatch coordi-

nator. As a result the scenario workshop walkthroughs did

not include representatives of the air services operator, and

no requirements on the dispatch coordinator were gener-

ated. In contrast, the walkthrough of one aircraft

turnaround scenario in the workplace took place in front of

the dispatch coordination office. After asking about the

office the facilitator negotiated access to it and asked

questions to the dispatch coordinator about their work,

problems, resources and needs. Some of the 12 require-

ments generated for the dispatch coordinator specified the

need to support effective local working practices, for

example the dispatch coordinator using VANTAGE shall

maintain important information about aircraft in a stats

sheet. Other requirements revealed basic problems with

airport operations that had consequences, not known

beforehand to the requirements process, on environmental

impact. One such problem was a lack of functioning radios

that meant that dispatchers could not communicate effec-

tively during aircraft turnarounds. The inference was that

working radios would improve the efficiency of aircraft

turnarounds and, as a result, contribute to aircraft

Table 4 The total number of requirements documented during each scenario by type

Scenario and

walkthrough type

AR FR LFR IR PR RR SR TR UR

VA2 (workshop) 0 17 0 0 8 0 1 0 2

VA3 (workshop) 0 10 0 0 6 0 0 0 0

VA1 (workshop) 1 26 0 0 2 2 0 0 1

VA11 (workshop) 0 10 0 0 1 0 0 0 1

VA4 (workplace) 9 28 0 1 6 1 0 1 13

AR availability, FR functional, LFR look-and-feel, IR inter-operabil- ity, PR performance, RR reliability, SR safety, TR training, UR usability

Table 5 Totals of requirements with subject actor, generated per VANTAGE scenario walkthrough

Requirement

subject actor

Scenario workshop

walkthroughs

Scenario

workplace

walkthrough

VA2 VA3 VA1 VA11 VA4

ATCO 3 1 7 2 0

BCA 0 0 0 0 3

VANTAGE system 14 8 16 4 19

Dispatcher 1 2 2 1 20

Ramp staff 2 1 2 1 1

Support services staff 2 0 0 0 0

Customer services agent 1 0 0 0 1

Dispatch coordinator 0 0 0 0 12

Pilot/aircraft 2 0 1 0 1

Airline 0 0 3 0 0

Airport operations staff 1 3 1 3 0

Passengers/general public 1 0 0 0 2

Stand guidance system 0 1 0 0 0

Other 1 0 0 0 0

98 Requirements Eng (2009) 14:91–111

123

movements that would reduce noise and gas emissions. The

facilitator generated requirements such as AR108 dis-

patchers, the dispatch coordinator the boarding staff shall

have sufficient communication resources to enable two-way

communication between these actors at all times.

4.4.2 Requirements themes

A second characteristic was the theme of each requirement.

The scenario workplace walkthroughs generated require-

ments with themes that were not generated during the

scenario workshop walkthroughs. Fourteen requirements

specified how VANTAGE should respond to bad weather

conditions. The walkthrough took place in bad weather

conditions depicted in Fig. 3. During a follow-up interview

the conditions prompted the facilitator to ask how they

affected aircraft turnaround. One dispatcher provided paper

documentation that specified bad weather restrictions on

turnaround equipment use, from which the facilitator

generated requirements such as AR112the VANTAGE sys-

tem shall support airport operations in all possible adverse

weather conditions that can arise at BCA and FR250the

VANTAGE system shall recognize the maximum operable

wind speed of 50 knots for stand parking of the A320/1

aircraft.

The scenario workplace walkthrough revealed another

theme—that dispatch work was highly mobile—which led

the facilitator to ask about requirements about mobile

working. One dispatcher reported previous experiences

with mobile tools at the nearby Belfast International Air-

port that revealed an opportunity for mobile computing for

dispatchers at BCA. An example availability requirement

was AR111the VANTAGE system shall connect with dif-

ferent mobile computing platforms at all airside locations.

Again, dispatchers in the scenario workshop walkthroughs

did not generate requirements on this theme.

4.4.3 Physical features in requirements

A third characteristic was description of geographical and

physical features of the airport in each requirement. We

might expect the scenario workplace walkthrough (in

which the analyst moves about the airport) to include more

references to geographical and physical features than

requirements generated during the workshops. Table 6

reports the totals of requirements that described geographic

features by scenario. Not surprisingly scenario VA4 On-

stand Operations, which was walked through in the

workplace, generated the largest number of such require-

ments. Some requirements make reference to the

complexities of taxiing and parking that emerges from the

topological layout of the airport, for example FR363 The

VANTAGE system shall include rules which incorporate

the specification of stands and FR369 The BCA tower

ATCO shall have access to information about the optimum

taxi route to allocated stand based on aircraft type and

loading. The walkthrough generated all of the requirements

that described air bridges, dispatch offices and departure

gates, for example AR106 BCA shall have sufficient airside

staff on duty at any one time to stand behind an aircraft

and stop road traffic during its pushback. The scenario

workshop walkthroughs did not generate requirements

describing the airport geography in the same level of detail.

4.5 The scenario workplace walkthrough

The facilitator and scribe adapted the VA4 On-stand

Operations scenario walkthrough to the workplace. It was

difficult to observe one aircraft turnaround from start to end

because more than one aircraft turned around at the same

time. Furthermore event timings and action durations were

unpredictable. Some scenario events related to a single

aircraft were simultaneous and described concurrent

actions that were difficult to observe, for example ramp

staff insert chocks, passenger steps and plug-in the aircraft

to ground power at the same time. Other actions lasted a

long time but revealed little to observe, for example the

crew clean the aircraft. Therefore the facilitator and scribe

walked through selected scenario events with one aircraft

before jumping to other scenario events with another

aircraft.

4.5.1 What triggered requirements generation

During the walkthrough the facilitator recognized more

event triggers to generate new requirements from the

workplace than from scenario events presented on the

MSP. For example requirement FR235 The dispatcher

shall be able to see and hear refueling activities on the

aircraft for which the dispatcher is responsible was gen-

erated in response to observations of a dispatcher’s reaction

Table 6 Totals of requirements that reference geographic features, generated per VANTAGE scenario walkthrough

Requirement

geographic

reference

Scenario workshop

walkthroughs

Scenario

workplace

walkthrough

VA2 VA3 VA1 VA11 VA4

Stand 8 11 3 1 7

Air bridge 0 0 0 0 2

Air side 0 0 0 1 8

Dispatch office 0 0 0 0 2

Departure gate 0 0 0 0 4

Road way 0 0 0 0 1

Requirements Eng (2009) 14:91–111 99

123

to the (loud) sound of the refueling truck stopping when

refueling stopped. We identified three possible reasons for

this dominance of workplace triggers. One was the richness

of the triggers in a complex and dynamic workplace such

as an airport. The facilitator was an experienced analyst

who drew on his experience to ask requirements discovery

questions in response to observed events that he knew the

VANTAGE system needed to respond to. A second reason

was the small screen size of the MSP. It was difficult for

the facilitator and scribe to read the MSP scenario at the

same time. Therefore, during the walkthrough, the facili-

tator and scribe developed a workaround. The scribe would

read out recognized events—typically alternative course

events—then the facilitator would investigate these events

as he was able. However, this workaround meant that the

more experienced facilitator could not browse and recog-

nize scenario events directly. A third possible reason was

the mismatch between the large number of alternative

course events generated in the scenario and browsed in the

MSP, and the relatively small number of real-world events

that might conceivably happen in the airport environment.

The effort needed by the scribe to scroll and read alterna-

tive course events in the MSP was greater than that needed

to observe the workplace. Hence the facilitator took the

simpler option during the walkthrough.

4.5.2 How requirements were documented

The procedure with which the facilitator specified

requirements also varied. In response to some events the

facilitator was able to document a requirement at the time

of the scenario event using information gathered from short

interactions with observed stakeholders. For example the

requirement AR105 BCA shall have sufficient airside staff

on duty at any one time not to delay the off-block time of

departing aircraft was generated in response to asking a

dispatcher why she had to stand next to aircraft during

pushback, and what would happen if she did not do it—an

interaction of no more than 20 s.

Other requirements could not be specified during the

scenario event because the event was too short, dangerous

or difficult, so the scribe marked the events to follow-up

when relevant stakeholders were available. Between

observations of scenario events, the facilitator conducted

structured interviews, sometimes lasting as long as 10 min,

with stakeholders to discuss observed events and document

stakeholder requirements. All were audio-recorded in full

using the MSP. For example, an interview with an expe-

rienced dispatcher led to requirement TR76 All dispatchers

who dispatch aircraft that load passengers using the air

bridge shall be trained to use aircraft cockpit instructions

to inform themselves of refueling status… based on observations of a busy dispatcher entering the aircraft

cockpit whilst embarking passengers. In one case—when

exploring requirements for dispatcher mobile working—

the facilitator prompted a mini-brainstorm, using the PDA

on which the MSP was running, to demonstrate possible

uses.

4.5.3 Scenario walkthrough productivity

The VANTAGE scenario walkthroughs consumed airport

and airline resources including pilots and air traffic con-

trollers. Therefore, we computed the estimates of

stakeholder time needed to generate a requirement in each

scenario workshop walkthrough and the scenario work-

place walkthrough. On average each of the four scenario

workshop walkthroughs involved eight stakeholders, lasted

2.5 h and generated 22 requirements. From this data we

compute that 1.1 requirements were generated per hour of

stakeholder participation. Stakeholder involvement in the

scenario workplace walkthrough was more difficult to

estimate. The walkthrough timetable was reviewed to

reveal that the walkthrough consumed approximately 7.2

person-h, including 5 h of time from the BCA environ-

mental manager who acted as a chaperone for the

walkthrough. The scenario workplace walkthrough gener-

ated 59 requirements. From this data, we compute that 8.2

requirements were generated per hour of stakeholder

involvement. The estimates, although crude, do suggest

that the VANTAGE scenario workplace walkthrough was

more productive in terms of stakeholder time.

5 Walking through scenarios in APOSDLE

We also walked through ART-SCENE scenarios to dis-

cover requirements for APOSDLE, a system to support

work-integrated learning by knowledge workers such as

aeronautic engineers, systems developers and consultants

working for chambers of commerce in Germany. The 4-

year project, funded by the European Commission, inclu-

ded application partners from the aerospace multinational

EADS, a German software development organization

called CNM, a business consultancy called ISN, and IHK,

the Darmstadt Chamber of Commerce.

Again we walked stakeholders through scenarios as part

of the RESCUE scenario-driven requirements process [18].

Prior to the walkthroughs the requirements team had dis-

covered requirements on the new APOSDLE system using

a creativity workshop and pair-wise use case authoring. A

use case model specified 15 core use cases that specified

how the future APOSDLE-enhanced work-integrated

learning should take place at different organizations.

Requirements on APOSDLE were associated with behav-

iour specified in the use cases. However, again, we still

100 Requirements Eng (2009) 14:91–111

123

needed APOSDLE stakeholders to discover complete

requirements on APOSDLE using projections of its future

use with scenario walkthroughs.

5.1 The scenario walkthrough schedule

APOSDLE stakeholders prioritized use cases for their

potential impact on work-integrated learning. The work-

shop walkthrough schedule is reported in Table 7. ART-

SCENE was used to generate one scenario for each of the

selected eight use cases. These eight scenarios were walked

through once in a scenario workshop walkthrough that was

attended by application and technology partners from dif-

ferent stakeholder partners. Selected scenarios were then

walked through again using the MSP with scenario work-

place walkthroughs at two application partner sites—ISN

and CNM. Six such scenario workplace walkthroughs took

place. Several scenarios, such as AP24 Use learning event

and AP12 Collaborate, were therefore walked through

three times—once in a workshop and twice in the work-

place at two application partner sites.

Figure 4 shows part of the ART-SCENE scenario for

AP24 Use learning event used in one of the scenario

workshop walkthroughs. Typical events included the

Table 7 The APOSDLE scenario walkthrough schedule

Date Scenario Type of

walkthrough

30/04/2007 AP8: monitor work context Workshop

01/05/2007 AP24: use learning event Workshop

01/05/2007 AP4a: find and contact relevant knowledge workers Workshop

02/05/2007 AP9b: store information exchanged Workshop

02/05/2007 AP12: collaborate Workshop

03/05/2007 AP6: make relevant knowledge artefact relevant to APOSDLE tools Workshop

04/05/2007 AP22: trigger learning Workshop

04/05/2007 AP23: construct and select learning event Workshop

10/05/2007 AP24: use learning event Workplace at ISN

10/05/2007 AP12: collaborate Workplace at ISN

16/05/2007 AP12: collaborate Workplace at CNM

16/05/2007 AP4a: find and contact relevant knowledge workers Workplace at CNM

16/05/2007 AP8: monitor work context Workplace at CNM

16/05/2007 AP24: use learning event Workplace at CNM

Fig. 4 One variation of the APOSDLE scenario (AP24 Use Learning Event) in the desktop version of ART-SCENE used

during one scenario workshop walkthrough

Requirements Eng (2009) 14:91–111 101

123

learning event is shown to the user and the user clicks on

the search refinement button. Each of the eight scenario

workshop walkthroughs took place in a meeting room with

one facilitator, one scribe who controlled ART-SCENE and

a second scribe who provided access to previous creativity

workshop results that were also displayed on a large screen.

The facilitator and stakeholders interacted directly with the

APOSDLE prototype displayed on an electronic white-

board through the touch screen. Each scenario workshop

walkthrough was designed to run for 2–4 h, depending on

the number of events in the scenario. Figure 5 shows a

facilitator interacting with and annotating the software

prototype during requirements discovery, and stakeholders

during the workshop surrounded by design prompts from

creativity workshop outcomes.

The six scenario workshop walkthroughs took place

over 2 days at the sites of two APOSDLE application

partners in Germany and Austria. One analyst, a native

German speaker who had not been present in the scenario

workshop walkthroughs, facilitated the scenario workplace

walkthroughs. A different scribe, also a German speaker,

operated the MSP. The facilitator observed work-based

learning behaviour and asked questions to staff being

observed. All spoken requirements and comments were

recorded in text and audio form in the MSP. Figure 6

shows the scenario workplace walkthrough at the consul-

tancy company ISN in Graz, Austria.

5.2 Generating the scenarios

Each generated scenario normal course specified the

expected order of events during APOSDLE’s generation,

use and management of learning material. For each nor-

mal course event in the eight scenarios, the algorithm

generated one or more candidate alternative courses for

each normal course event. This time alternative courses

were generated using the domain-independent ART-

SCENE algorithm [17] extended with a domain-specific

version that included class hierarchies of abnormal

behaviour and state derived from published learning

Fig. 5 Images from one APOSDLE scenario workshop walkthrough, the left-hand side showing annotation of software

prototypes on an electronic

whiteboard, the right-hand side showing stakeholders

surrounded by outputs from the

earlier creativity workshop

Fig. 6 An APOSDLE scenario workplace walkthrough, showing the use of the MSP at

consultancy organization ISN

102 Requirements Eng (2009) 14:91–111

123

literature. Domain-independent alternative courses inclu-

ded what if the knowledge worker lacks the necessary

knowledge? and what if the expert exhibits some abnor-

mal behaviour? Domain-dependent alternative courses

included what if the user’s environment is inappropriate

for learning? and what if the communication medium used

is inappropriate? Other examples are shown on the right-

hand side of Fig. 4. Each generated scenario specified on

average 13 normal course events and 19 alternative course

events per normal course event. The shortest scenario was

AP6: Make relevant knowledge artefact relevant to

APOSDLE tools with five normal course and a total of 78

alternative course events. The longest was AP24 Use

learning event with 19 normal course and a total of 457

alternative course events in the principal normal course

and four variation scenarios.

5.3 Scenario walkthrough results

All scenario walkthroughs took place as planned. Each of

the eight scenario workshop walkthroughs lasted between 2

and 4 h. Between three and eight technology and end-user

stakeholders attended each workshop, and each included at

least one technology partner and one application partner.

The six scenario workplace walkthroughs lasted a total of

about 10 h not including time taken for tool setup and

coffee breaks. The mobile analysts observed and interacted

with end-users from the application partners rather than the

technology developers.

Results from the scenario walkthroughs are reported in

Table 8. The eight scenario workshop walkthroughs

generated 228 requirements that included 46 requirements

for AP24 Use learning event. The average number of

requirements generated per scenario workshop walk-

through was 28.5. As in VANTAGE these requirements

were documented using ART-SCENE. Scenario work-

place walkthroughs generated 160 requirements generated

mostly during walkthroughs of scenarios AP12 and AP24.

The scenario workplace walkthroughs at ISN in Graz

discovered 52 requirements for AP12 and 19 require-

ments for AP24. At CNM in Dortmund the scenario

workplace walkthroughs generated 24 requirements for

AP12 and 36 requirements for AP24. Additionally, at

CNM we gathered requirements for two other scenarios,

which were not the focus of the inquiry, so less time was

spent walking through them. The average number of

requirements generated per scenario workplace walk-

through was 26.7.

Requirements were again analyzed by type as reported

in Table 9. Results revealed that, unlike VANTAGE, the

scenario workshop walkthroughs generated more usability-

, performance- and maintainability-type requirements than

did the scenario workplace walkthroughs. In contrast the

scenario workplace walkthroughs generated 18 security-

Table 8 The total number of requirements documented during each scenario, types of event for which requirements were documented, and average number of requirements generated per scenario normal course event, for the APOSDLE project

Scenario and walkthrough type Total number of

requirements

documented

Number of requirements

on use case and normal

course event behaviour

Number of

requirements

on alternative

course behaviour

Average number of

requirements per

normal course event

AP8: monitor work context (workshop) 36 25 11 4.0

AP24: use learning event (workshop) 46 32 14 2.0

AP4a: find and contact relevant knowledge workers

(workshop)

30 28 2 3.0

AP9b: store information exchanged (workshop) 32 20 12 4.0

AP12: collaborate (workshop) 20 16 4 1.82

AP6: make relevant knowledge artefact relevant

to APOSDLE tools (workshop)

10 10 0 2.0

AP22: trigger learning (workshop) 22 18 4 0.81

AP23: construct and select learning event (workshop) 32 27 5 4.57

AP12: collaborate (workplace at ISN) 52 44 8 2.73

AP24: use learning event (workplace at ISN) 19 13 6 1.72

AP12: collaborate (workplace at CNM) 24 21 3 2.18

AP4a: find and contact relevant knowledge workers

(workplace at CNM)

7 5 2 0.7

AP8: monitor work context (workplace at CNM) 22 20 2 2.75

AP24: use learning event (workplace at CNM) 36 30 6 1.89

Total 388 309 79

Requirements Eng (2009) 14:91–111 103

123

type and 11 interoperability-type requirements, in contrast

to low numbers of these requirement type generated in the

scenario workshop walkthroughs. However, no overall

pattern of requirement generation by type emerged.

Although most of the 228 requirements generated in the

scenario workshop walkthroughs were expressed in text

form, the screen capture capability of the electronic

whiteboard enabled the facilitator and stakeholders to

annotate and save images of the software prototype asso-

ciated with these requirements. Fourteen of these 228

(6.1%) APOSDLE requirements were documented in this

manner. Figure 7 shows three examples of these enhanced

requirement descriptions.

The six scenario workplace walkthroughs generated a

total of 58 observations (an average of 9.67 per scenario)

recorded as comments. Of these 58 comments, 31 were

documented for AP12 Collaborate at the CNM site where

observation time was significantly higher than the time

spent interviewing stakeholders during the walkthroughs.

5.4 Qualitative requirements analysis

We investigated the APOSDLE requirements to detect

qualitative differences between requirements based on

requirement subject, theme and inclusion of physical fea-

tures generated with the different types of scenario

walkthrough. To enable this investigation we first analyzed

the requirements to remove duplicates that arose from

walking through the same scenario multiple times with

different stakeholders. Of the 160 requirements generated

during scenario workplace walkthroughs, 22 had already

been specified once before in a walkthrough and 1 had been

specified twice before. Of the 22 duplicated requirements,

12 had been originally specified in scenario workshop

walkthroughs and 10 in scenario workplace walkthroughs.

The removal of these 24 repeating requirements resulted in

a total of 364 unique APOSDLE requirements that were

investigated for their subjects, themes and description of

physical features in the work context.

Table 9 The total number of requirements documented during each scenario by scenario walkthrough type

Scenario AR FR LFR IR PR RR SR TR UR BUS LG MR DR

Scenario workshop

walkthroughs

AP8 0 23 0 1 2 1 0 0 4 2 1 2 0

AP4 0 26 1 0 0 0 0 0 15 3 0 0 1

AP4a 0 30 0 0 0 0 0 0 0 0 0 0 0

AP9 0 29 0 0 0 0 0 0 2 0 0 1 0

AP12 0 18 0 1 0 0 0 1 0 0 0 0 0

AP6 0 7 0 0 0 0 0 0 3 0 0 0 0

AP22 0 14 0 0 2 0 0 0 5 0 0 1 0

AP23 0 18 0 0 0 1 0 0 12 0 0 1 0

Scenario workplace

walkthroughs at ISN

AP24 0 17 0 0 0 0 2 0 0 0 0 0 0

AP12 0 28 0 7 0 2 5 1 2 7 0 0 0

Scenario workplace

walkthroughs at CNM

AP12 0 14 0 3 0 0 5 0 0 2 0 0 0

AP4a 0 6 0 0 0 0 0 0 0 1 0 0 0

AP8 0 13 0 0 0 0 6 0 1 1 1 0 0

AP24 0 28 0 1 0 0 0 0 1 6 0 0 0

AR availability, FR functional, LFR look-and-feel, IR inter-operability, PR performance, RR reliability, SR safety, TR training, UR usability, BUS business goals, LG legal, MR maintainability, DR device

Fig. 7 Three APOSDLE screen shots taken from requirements discovered during the AP12 Collaborate, AP24 Use learning event and AP4a Find and contact relevant knowledge workersscenario workshop walkthroughs

104 Requirements Eng (2009) 14:91–111

123

5.4.1 Requirements subjects

The first characteristic was the subject of each requirement.

As in VANTAGE, ART-SCENE mandated that all

requirements were expressed using shall statements with a

common structure [3] that highlighted the subjects of

requirements as the actor upon which the requirement was

specified. Results are reported in Table 10. Most require-

ments were on the APOSDLE system and its users

independent of the type of walkthrough that generated

them (e.g., The APOSDLE system shall generate a learning

goal from the user question). Compared to VANTAGE,

there were fewer differences regarding requirement sub-

jects between scenario workshop walkthroughs and

scenario workplace walkthroughs. The scenario workplace

walkthroughs generated requirements on the learner, col-

laboration transcript and collaboration document not

generated in the scenario workshop walkthroughs.

5.4.2 Requirements themes

A second characteristic was the theme of each requirement

reported in Table 11. There was one main requirement

theme generated during the scenario workplace walk-

throughs—privacy. Although eight requirements with this

theme had been generated in scenario workshop walk-

throughs, the scenario workplace walkthroughs generated

17 additional privacy-type requirements such as the

APOSDLE system shall delete context monitoring history

after a short time. However, compared to the VANTAGE

Table 10 Totals of requirements with different subject actors, generated per APOSDLE scenario walkthrough

Requirements subject actor Scenario workshops walkthroughs Scenario workplace

walkthroughs at ISN

Scenario workplace

walkthroughs at CNM

AP8 AP24 AP4a AP9 AP12 AP6 AP22 AP12 AP24 AP12 AP4a AP8 AP24

APOSDLE system 22 32 18 12 8 7 20 26 8 14 4 18 23

Collaboration participant 0 0 0 3 1 0 0 5 0 1 0 0 0

User 12 13 6 17 8 3 2 8 2 1 0 1 9

Expert 0 1 6 0 1 0 0 0 0 0 1 0 0

Collaboration transcript 0 0 0 0 0 0 0 3 0 0 0 0 0

Collaboration document 0 0 0 0 0 0 0 3 0 0 0 0 0

Learner 0 0 0 0 0 0 0 2 6 0 0 0 0

Customer 0 0 0 0 0 0 0 0 0 1 0 0 0

Administrator 1 0 0 0 0 0 0 0 0 0 0 0 0

Collaboration tool 0 0 0 0 2 0 0 0 0 0 0 0 0

Knowledge engineer 1 0 0 0 0 0 0 0 0 0 0 0 0

Table 11 Totals of requirements by requirements themes, generated per APOSDLE scenario walkthrough

Requirements themes Scenario walkthrough workshops Scenario workplace

walkthroughs at ISN

Scenario workplace

walkthroughs at CNM

AP8 AP24 AP4a AP9 AP12 AP6 AP22 AP12 AP24 AP12 AP4a AP8 AP24

APOSDLE affect on users 4 3 0 0 0 0 4 8 0 0 1 1 2

APOSDLE affect on current work practices 2 3 0 0 0 0 1 4 0 0 0 1 2

APOSDLE support for mobility 0 2 0 0 0 0 0 3 0 0 0 0 0

APOSDLE user profile 2 0 9 2 1 0 5 0 1 0 1 0 2

Knowledge artefact 1 10 0 19 2 4 0 5 0 0 0 0 0

Context monitoring 14 0 0 0 1 0 1 2 0 1 0 4 0

Privacy issues 2 1 1 3 1 0 0 6 2 4 0 5 0

Create knowledge artefact 1 1 0 3 3 0 1 4 1 7 0 5 1

Availability for collaboration 0 1 4 0 0 0 0 2 7 0 3 3 0

Learning process 4 23 0 0 1 5 9 3 4 0 0 0 24

Collaboration process 0 2 16 3 10 0 0 7 0 5 0 0 0

Other 0 0 0 0 0 1 0 3 1 0 0 0 1

System administration 4 0 0 2 1 0 1 0 0 0 0 0 0

Requirements Eng (2009) 14:91–111 105

123

project, there were fewer differences in the themes of

requirements generated in the scenario workshop walk-

throughs and scenario workplace walkthroughs.

5.4.3 Physical features in requirements

The third characteristic was the description of physical

features of the ISN and CNM office in each requirement. In

contrast to the VANTAGE project, we did not discover

requirements that referred to any physical features. One

possible reason is that physical features are less important

for a desktop based learning support system than for an

airport management system.

5.5 Scenario walkthrough productivity

Again we computed the estimates of APOSDLE stakeholder

time needed to generate a requirement in a scenario workshop

walkthrough and a scenario workplace walkthrough. On

average each of the seven workshops involved 3.9 stakeholders

(not including facilitator and scribe), lasted 2.45 h and gen-

erated 28.4 requirements. From this data we compute almost

3.0 requirements were generated per hour of stakeholder par-

ticipation, which was higher than the rate of generation during

the four VANTAGE scenario workshop walkthroughs.

Calculating stakeholder time spent on the APOSDLE

scenario workplace walkthroughs was again more difficult.

At ISN stakeholders participated in walkthroughs that lasted

a total of 3.8 h. We generated 71 requirements including the

eight duplicate ones. Both analysts interacted with the

stakeholders for 2.8 h—the remainder of the time was spent

observing them. From this data, we computed 13.2 require-

ments were generated per hour of stakeholder participation,

higher than the rate of the VANTAGE scenario workplace

walkthroughs. At CNM we spent 3.2 h (55%) on interactions

with stakeholders, while 2.6 h (45%) were spent on obser-

vations. During the scenario workplace walkthroughs at

CNM 89 requirements were generated (73 without dupli-

cates). We computed that 17.7 requirements were generated

per hour of stakeholder participation, again a rate higher than

the VANTAGE scenario workplace walkthroughs.

Furthermore, on average, the scenario workshop walk-

throughs generated 5.8 requirements per hour of analyst

and scribe participation. Scenario workplace walkthroughs

at ISN generated 8.3 requirements per hour of analyst

participation. Due to the increased observation time at

CNM the scenario workplace walkthrough generated 6.2

requirements per hour.

5.6 The scenario workplace walkthroughs

Most tasks that were walked through in the workplace were

standard office tasks (e.g., checking e-mail) that could be

mapped to scenario normal course events. As a conse-

quence analysts were able to interrupt stakeholders to ask

questions as well as ask follow-up questions after unin-

terruptible tasks (e.g., talking to a customer on the phone).

The facilitator and scribe rotated the roles to increase their

chances of recognizing cues provided by the MSP. Over

time both became familiar with the scenarios, which

reduced time to navigate them.

5.6.1 What triggered requirements generation?

After the scenario workplace walkthroughs the two ana-

lysts reflected that most requirements were triggered by

events in the workplace, for two reasons. The first was that,

during APOSDLE, both the analyst and scribe were

equipped with the MSP tool, which allowed them to read

the scenarios at the same time. The second was that the

office environment was simpler, less dynamic and therefore

provided fewer triggers than the airport environment.

5.6.2 How requirements were documented

In contrast to VANTAGE most requirements were docu-

mented in text rather than audio form using the MSP stylus

and keyboard. One reason was that, because the analyst and

scribe were equipped with PDAs, there was no need to

communicate scenario information to the facilitator, which

in turn gave more freedom to the scribe. The less dynamic

and mobile office environment also gave the analysts more

time to type requirements into the MSP, a luxury not

available in previous scenario workplace walkthroughs

with the MSP [19]. During one scenario workplace walk-

through the scribe even replaced the MSP with the desktop

Scenario Presenter running on a notebook computer to

document requirements due to the time available and

chance to sit at a desk.

6 The research questions revisited

The scenario walkthroughs in VANTAGE and APOSDLE

were a success. They led to generation of 147 and 338 new

requirements, respectively. The use of ART-SCENE was

also a success, in that we effectively applied a research

prototype to two challenging requirements problems. We

extended the use of the desktop Scenario Presenter in

facilitated scenario workshop walkthroughs to support

software prototype walkthroughs. The use of the MSP in

VANTAGE is one of the first reported effective uses of

mobile requirements tools on large projects [20]. Simple-

to-use audio recording of spoken requirements overcame

the usability problems reported in [19], whilst giving the

MSP to experienced analysts realized its potential in

106 Requirements Eng (2009) 14:91–111

123

different settings. That said, problems remained, such as

difficulties encountered by two people browsing scenario

events with a single MSP. We reviewed the VANTAGE

and APOSDLE results to answer the three research

questions.

6.1 Effect on scenario walkthroughs from a software

prototype?

The answer to question Q1—does a workshop walkthrough

of a scenario supported with a software prototype and

creativity prompts generate more requirements—is a ten-

tative yes based on data from the scenario workshop

walkthroughs. Not only did the APOSDLE scenario

workshop walkthroughs with the software prototype gen-

erate more requirements per scenario—28.5 to 26.7—than

the VANTAGE scenario workshop walkthroughs, but the

APOSDLE scenarios were shorter. In APOSDLE 2.51

requirements were generated per normal course event, as

opposed to 1.05 requirements per VANTAGE scenario.

Fourteen APOSDLE requirements were supported with

annotated screenshots and images not available in VAN-

TAGE due to the absence of a prototype.

The quantitative results provide preliminary evidence

that software prototypes can provide additional recognition

cues with which to generate and specify new requirements.

However, the answer does need to be interpreted with care

as other variables such as the domain, degrees of stake-

holder participation and expertise, and requirements

specified previously in the process clearly may all have

influenced the result. Threats to validity of the findings are

discussed later.

6.2 Different requirements from scenario workplace

walkthroughs?

The answer to question Q2—does walking through ART-

SCENE scenarios in the workplace lead to generation of

different requirements to workshops—is also a tentative

yes. One scenario workplace walkthrough generated

requirements on VANTAGE actors that the scenario

workshop walkthroughs did not generate requirements on.

It acquired requirements from new VANTAGE stake-

holders not identified during earlier analyses. It acquired

requirements from stakeholders who had attended the

scenario workshop walkthroughs but not specified these

requirements during them. And it generated requirements

of different types on important themes not identified during

the VANTAGE scenario workshop walkthroughs. In

APOSDLE the scenario workplace walkthroughs at two

different sites revealed different, potentially conflicting

requirements that did not emerge clearly during the earlier

scenario workshop walkthroughs. These walkthroughs also

generated important observations not captured during the

scenario workshop walkthroughs related to the require-

ments that were specified.

There are several possible reasons for these results. The

first is the use of scenario workplace walkthroughs to do a

stakeholder analysis—discovering then involving all of the

important actors in VANTAGE. This was true for the

dispatch coordinator role. A second possible reason was

that observing the workplace enabled the facilitator to act

as an apprentice and learn about actors’ work, as supported

in contextual inquiry [6]. Indeed, some periods of the

scenario workplace walkthrough were indeed a form of

ethnographic observation, albeit structured using the sce-

nario in the MSP. This learning then enabled the facilitator

to ask more informed questions during observations and

structured interviews, as well as to infer more correct and

complete VANTAGE requirements. In contrast the facili-

tator and scribe in the scenario workshop walkthroughs

often did not have access to the domain knowledge needed

to complete the specification of requirements because

knowledge about the workplace was not available to them.

A third possible reason—an important one—is that the

workplace provided different event recognition cues to

discover requirements on different themes. Indeed, rather

than trigger event recognition, the facilitator used the MSP

scenario in the scenario workplace walkthroughs primarily

to generate requirements and requirements-related data in

the context of the observed normal course event. This had

two important advantages. The first is that related

requirements and material could be reviewed during and

between walkthroughs, thus enabling the facilitator to ask

more informed structured interview questions. The second

is that, during post-walkthrough analyses, analysts could

review the requirements and related material in context,

thus providing cues to recall the observed event and more

information with which to infer new requirements.

6.3 More requirements from scenario workplace

walkthroughs?

The answer to question Q3—does walking through ART-

SCENE scenarios in the workplace lead to generation of

more requirements than in workshops—is also a tentative

yes. The one VANTAGE scenario workplace walkthrough

generated a larger number of requirements than any single

scenario workshop walkthrough. All VANTAGE and

APOSDLE scenario workplace walkthroughs were more

productive in terms of stakeholder time than the workshop

equivalent. The repeated APOSDLE scenario workplace

walkthroughs also generated new requirements not

described in the scenario workshop walkthroughs.

Of course repeating walkthroughs of the same scenario

in different workplaces risked the duplication of

Requirements Eng (2009) 14:91–111 107

123

requirements that needed significant analyst effort to detect

and remove. Although just over 18% of the APOSDLE

requirements generated during the scenario workplace

walkthroughs were semantic duplicates that needed to be

removed from the requirements specification, over 80% of

the requirements from walking through the same scenario a

second or third time were new to the process, providing

results with which to answer yes to the research question.

There are at least two possible reasons for the greater

productivity of the VANTAGE scenario workplace walk-

through. The first is the role of the facilitators who, in the

workplace, directly inferred and documented more

requirements than in the workshops because of the limited

communication that was possible with stakeholders

engaged in other tasks. The outcome was that the facili-

tators were able to infer more requirements than they were

able to acquire from stakeholders in the workshops. This

has implications for redesigning the scenario workshop

walkthroughs to enable the facilitators to infer and propose

new requirements.

Conversely, a second reason is that the scenario work-

place walkthroughs increased the requirements

communication bandwidth. Whereas stakeholders did not

bring written material to the workshops, the facilitators in

the workplace were able to collect documents such as the

bad weather operation document, as well as observe

workplace artifacts such the dispatch coordinator’s stats

sheet and take photographs. This material added to the

spoken requirements recorded in the MSP and provided a

richer data corpus that the analyst used to infer larger

numbers of requirements than in the workshops. Again this

raises the need to design scenario workshop walkthroughs

to encourage inference of requirements from different

information sources.

6.4 Threats to validity

We report the pragmatic use of different scenario walk-

through types to solve the VANTAGE and APOSDLE

requirements problems. Our decision not to balance inde-

pendent variables and control dependent ones across a low

number of walkthroughs means that all results need to be

interpreted with care. For example, the effectiveness of the

scenario workplace walkthroughs could also have been

influenced by the design of the walkthroughs, the stake-

holder participation in them, and the reporting of the

results. The VANTAGE and APOSDLE scenario work-

place walkthroughs always occurred after the scenario

workshop walkthroughs, hence facilitator behaviour might

have been informed by domain knowledge obtained in the

earlier workshop walkthroughs. The influence on stake-

holders was less because, in both projects, most

stakeholders observed in the scenario workplace

walkthroughs did not participate in the scenario workshop

walkthroughs. Implicit biases might also have risen from

the desire of the facilitator and scribe to see the scenario

workplace walkthrough succeed, especially in light of

problems reported in earlier uses [19]. However, the effort

needed to set up and run the scenario walkthroughs under

challenging conditions in the VANTAGE and APOSDLE

projects, we believe, reduced the likelihood of such

implicit bias due to the analyst’s focus on more undertak-

ing the requirements tasks making both types of

walkthrough succeed to their best abilities.

7 Lessons learned

The following lessons were learned about the design and

running of scenario walkthroughs in requirements projects

from our experiences in APOSDLE and VANTAGE.

The most obvious lesson is that mixing and matching

different types of scenario walkthroughs, in our case sce-

nario walkthroughs in facilitated workshops and in the

workplace, generated more requirements on different

actors and about different themes. In simple terms, differ-

ent scenario walkthroughs increased the completeness of

resulting requirements specification over sole use of one

walkthrough type. A related lesson was to walkthrough the

same scenarios more than once with different stakeholders.

Although this led to duplicate requirements being specified,

over four in every five requirements generated during the

walkthroughs were original and valid. Some requirements

duplication may be an acceptable price for ensuring more

complete requirements specification.

One unexpected outcome of the VANTAGE scenario

workplace walkthrough was the discovery of one new

stakeholder with important requirements on the new sys-

tem. It provides direct evidence for the effectiveness of

scenario workplace walkthroughs for stakeholder analysis.

Although stakeholder analysis techniques are available

[16], scenario workplace walkthroughs earlier in the

requirements process, using simple scenarios that outline

key events without exploring alternative course events, can

complement existing analysis techniques and validate a

current stakeholder model. Such walkthroughs earlier in

the requirements process can make the scenarios more

complete for later walkthroughs in workshops, as results

showed that scenarios were edited and commented more

frequently in the scenario workplace walkthroughs.

Results also provide lessons for designing scenario

walkthroughs to be more effective. In particular the

VANTAGE scenario workplace walkthrough allowed the

analyst to generate new requirements that stakeholders

later accepted, in strong contrast to scenario workshop

walkthroughs in which the analyst encouraged stakeholders

108 Requirements Eng (2009) 14:91–111

123

to generate requirements. The workshop walkthrough

process has been extended to provide periods in which the

analyst can propose speculative new requirements to be

accepted or rejected by the stakeholders present. The sce-

nario workplace walkthroughs in both projects

demonstrated the value of interleaving the walkthrough

with more detailed stakeholder interviews and analysis of

documentation available in the workplace, although this

was not part of the original walkthrough protocol. The

protocol has been changed to allow for documentation

collection, brainstorming and interview sessions, and

guidelines given to stakeholders before scenario workshop

walkthroughs have been extended to encourage stake-

holders to bring relevant documentation to workshops.

Furthermore, results indicate that different types of

scenario walkthrough with different types of stakeholders,

sometimes in different workplaces, can influence the sub-

jects and themes of the generated requirements. We

recommend more a priori design of scenario walkthrough

schedules that takes into account the acquisition of sets of

requirements using information about the workplace and

stakeholders. A requirements framework that structures

requirements by subject and theme can inform the design

of such a schedule.

Results from the APOSDLE scenario workshop walk-

throughs indicated another unexpected lesson. Whilst the

walkthroughs revealed weak evidence that the software

prototype and creativity cues might have increased the

number of requirements specified, one expected outcome

was annotation of the prototype to illustrate requirements

graphically using electronic whiteboards. Such illustrations

can communicate requirements to stakeholders and

designers more effectively, as well as lead to more

requirements generation in walkthroughs.

Another lesson emerges from the productivity results.

Scenario workplace walkthroughs were more efficient in

terms of stakeholder participation time to generate

requirements. If time is short, we recommend running more

scenario workplace walkthroughs rather than scenario

workshop walkthroughs.

One final lesson relates to the scenarios walked through

in both projects. Stakeholders lacked the time needed to

walk through alternative course events automatically gen-

erated by ART-SCENE. Although results do not indicate

that this has led to requirements incompleteness in both

projects, analysts perhaps need to specify and generate

scenarios with fewer normal course events, to allow more

effective walking through of the normal and alternative

course events.

To conclude the lessons indicate some comparative

strengths and weaknesses of scenario workshop walk-

throughs and scenario workplace walkthroughs when

supported with different versions of the ART-SCENE

scenario environment. The next section places the strengths

of scenario workplace walkthroughs in a wider context.

8 Related work

Ethnographical methods have been used in requirements

projects to provide an adequate understanding of the cur-

rent work practice to be changed by specified systems.

Several researchers used ethnographical methods to inform

requirements engineering in various domains including air

traffic control [5] and underground control rooms [11]. In

some case ethnographical methods were combined with

existing requirements techniques, such as viewpoints to

structure the results of an ethnographic study [13]. Viller

and Sommerville [27] reported different uses of ethno-

graphical methods during requirements processes.

Contextual inquiry is an approach influenced by eth-

nography that supports system development. In contrast to

other ethnographic methods, an analyst with a technical

background is in charge of analyzing existing work prac-

tice [7, 29]. Contextual inquiry is based on observation and

the contextual interview, the key activity to gather design

relevant data in the stakeholders’ work environment. The

interview is structured following the principles of contex-

tual inquiry [6]. Contextual inquiry has been successfully

applied in various projects in the software engineering

domain [6]. Holtzblatt [12] concludes that building a

design upon field data was essential for the success of these

projects.

Ethnographical methods and contextual inquiry support

analysts’ understanding of the workplace. However, prob-

lems have been highlighted [13, 22, 27]. Most still use a

paper and pencil-based approach and lack on-site tool

support for guiding on-site analysts and for documenting

the gathered information. Contextual inquiry techniques

are only weakly integrated with existing requirements

methods and tools. Moreover the volume of information

gathered is often unfocused, which makes the information

difficult to use in the requirements process. There is also a

lack of a theoretical structure underpinning the observation

process [22] and due to a lack of focus these approaches are

confined to relatively small-scale environments (e.g., con-

trol rooms) [14]. The introduction of mobile tools for

walking through scenarios in the workplace reported in this

paper was designed to overcome some of these reported

problems.

9 Future research

Future research is in three directions. The first will extend

the model of scenario-based requirements discovery with

Requirements Eng (2009) 14:91–111 109

123

new physical tasks such as perceive recognition cues in the

work context and perceive possible design features, and

cognitive tasks such as infer new requirement, which will

relate to models of creativity in requirements engineering.

Secondly, we will then apply the model to redesign the

scenario workshops to support analysts to infer candidate

requirements and propose them to stakeholders. ART-

SCENE will be extended with pattern-based requirements

generation that can recommend outline requirements

automatically. We will also build on existing methods [24]

to develop new walkthrough processes, techniques and

protocols to manage the effective use of scenario proto-

types that provide effective additional recognition cues for

discovering requirements during facilitated workshops.

Whilst MSP audio recording of spoken requirements

overcame earlier usability problems, its use here revealed

new challenges to solve. One is the provision of scenario

event cues to both the facilitator and scribe as we explored

in the reported APOSDLE walkthroughs. Screen size is

dictated by available PDA devices, so one solution is to

synchronize scenario walkthroughs on two devices. Whilst

the scribe navigates the scenario on one device running the

current MSP, a selected subset of scenario events, for

example one normal course event and alternative course

events associated with it, are displayed on the second

device to provide the facilitator with manageable, context-

specific event recognition cues. Analysts could also use this

feature to select between generated alternative course

events to present to the facilitator, to reduce information

overload. One possible further refinement is to use context-

aware devices to filter scenario events dynamically

according to proximity to a location or actor. We look

forward to reporting these outcomes in the near future.

Acknowledgments Work reported in this paper was funded in part by the UK DTI-funded VANTAGE Phase-1 project and in part by the EU-funded FP6 027023 APOSDLE project.

References

1. Agentsheets web site: http://agentsheets.com/

2. Alexander IF, Maiden NAM (eds) (2004) Scenarios, stories and

use cases. John Wiley, New York

3. Alexander IF, Stevens R (2002) Writing better requirements.

Addison-Wesley, Reading

4. Baddeley AD (1990) Human memory: theory and practice.

Lawrence Erlbaum Associates, Mahwah

5. Bentley R, Hughes JA, Randall D, Rodden T, Sawyer P, Shapiro

D, Sommerville I (1992) Ethnographically-informed systems

design for air traffic control. In: Proceedings ACM conference on

computer supported cooperative work (CSCW), pp 123–129

6. Beyer H, Holtzblatt K (1998) Contextual design: defining con-

sumer-centered systems. Morgan-Kauffman, San Francisco

7. Blomberg J, Burrell M, Guest G (2002) An ethnographic

approach to design. In: Jacko JA, Sears A (eds) The human–

computer interaction handbook: fundamentals, evolving

technologies and emerging applications. Lawrence Erlbaum

Associates, Mahwah, pp 964–986

8. Carroll JM (2000) Making use: scenario-based design of human–

computer interactions. MIT Press, Cambridge

9. Gottensdeiner E (2004) Running a use case/scenario workshop.

In: Alexander I, Maiden NAM (eds) Scenarios, stories, use cases:

through the systems development life-cycle. John Wiley, New

York, pp 81–101

10. Haumer P, Heymans P, Jarke M, Pohl K (1999) Bridging the gap

between past and future in re: a scenario-based approach. In:

Proceedings of the 4th IEEE international symposium on require-

ments engineering. IEEE Computer Society Press, pp 66–73

11. Heath C, Luff P (1992) Crisis management and multimedia

technology in London underground line control rooms. J Comput

Support Cooperative Work 1(1):24–48

12. Holtzblatt K (2004) The role of scenarios in contextual design:

from user observations to work redesign to use cases. In: Alex-

ander IF, Maiden N (eds) Scenarios, stories, use cases: through

the systems development life-cycle. John Wiley & Sons, New

York, pp 179–209

13. Hughes J, King V, Rodden T, Andersen H (1995) The role of

ethnography in interactive systems design. Cooperative Systems

Engineering Group, Lancaster University, Technical report

CSEG/8/1995

14. Hughes J, King V, Rodden T, Andersen H (1994) Moving out

from the control room: ethnography in system design. In: Pro-

ceedings of the ACM conference on computer supported

cooperative work (CSCW), pp 429–439

15. Jones SV, Lynch P, Maiden NAM, Lindstaedt S (2008) Use and

influence of creative ideas and requirements for a work-integrated

learning system. In Proceedings 16th IEEE international confer-

ence on requirements engineering. IEEE Computer Society Press

16. Macaulay L (1993) Requirements capture as a cooperative

activity. In: Proceedings of the IEEE international symposium on

requirements engineering. IEEE Computer Science Press,

pp 174–181

17. Maiden NAM (2004) Systematic scenario walkthroughs with

ART-SCENE. In: Alexander I, Maiden NAM (eds) Scenarios,

stories, use cases : through the systems development life-cycle.

John Wiley, New York, pp 161–178

18. Maiden NAM, Jones SV, Manning S, Greenwood J, Renou L

(2004) Model-driven requirements engineering: synchronising

models in an air traffic management case study. In: Proceedings

of CaiSE’2004. LNCS, vol 3084, pp 368–383. Springer, Berlin

19. Maiden NAM, Seyff N, Grunbacher P, Otojare O, Mitteregger K

(2006) Making mobile requirements engineering tools usable and

useful. In: Proceedings of the 14th IEEE international conference

on requirements engineering. IEEE Computer Society Press

20. Maiden NAM, Seyff N, Grunbacher P, Otojare O, Mitteregger K

(2007) Determining stakeholder needs in the workplace. IEEE

Softw 27(2):46–52

21. Mavin A, Maiden NAM (2003) Determining socio-technical

systems requirements: experiences with generating and walking

through scenarios. In: Proceedings of the 11th IEEE international

conference on requirements engineering. IEEE Computer Society

Press

22. Maxwell C, Millard N (1999) Integrating ethnographic field

observations into requirements engineering. http://www.comp.

lancs.ac.uk/computing/research/cseg/projects/coherence/

workshop/Maxwell.html. Workshop—an industrial approach to

work analysis and software design. http://www.comp.lancs.ac.

uk/computing/research/cseg/projects/coherence/workshop.html

23. Robertson S, Robertson J (1999) Mastering the requirements

process. Addison-Wesley, Longman, Reading, London

24. Sutcliffe AG (1997) A technique combination approach to

requirements engineering. In: Proceedings of the 3rd international

110 Requirements Eng (2009) 14:91–111

123

symposium on requirements engineering. IEEE Computer Soci-

ety Press

25. Sutcliffe AG, Maiden NAM, Minocha S, Manuel D (1998)

Supporting scenario-based requirements engineering. IEEE Trans

Softw Eng 24(12):1072–1088

26. Uchitel S, Chatley R, Kramer J, Magee J (2004) Fluent-based

animation: exploiting the relationship between goals and sce-

narios for requirements validation. In: Proceedings of the 12th

international IEEE requirements engineering conference. IEEE

Computer Society, pp 208–217

27. Viller S, Sommerville I (1999) Social analysis in the require-

ments engineering process: from ethnography to method. In:

Proceedings of the IEEE international symposium on require-

ments engineering, pp 6–13

28. Weidenhaupt K, Pohl K, Jarke M, Haumer P (1998) Scenario

usage in systems development: a report on current practice. IEEE

Softw 15(2):34–45

29. Whiteside J, Wixon D (1988) Contextualism as a world view for

the reformation of meetings. In: Proceedings of the ACM con-

ference on computer-supported cooperative work (CSCW),

pp 369–376

30. Zachos K, Maiden NAM, Tosar A (2005) Rich media scenarios

for discovering requirements. IEEE Softw 22(5):89–97

Requirements Eng (2009) 14:91–111 111

123