Application 1 – Analysis and Synthesis of Prior Research
tchyar
Anexploratorystudyofarchitecturaleffectsonrequirementsdecisions.pdf
A
J D
a
A R R A A
K S R E S P Q A
1
fi o e r t W t s t f T 1 W e r r e a
(
t
0 d
The Journal of Systems and Software 83 (2010) 2441–2455
Contents lists available at ScienceDirect
The Journal of Systems and Software
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / j s s
n exploratory study of architectural effects on requirements decisions�
ames A. Miller, Remo Ferrari ∗, Nazim H. Madhavji ∗
epartment of Computer Science, University of Western Ontario, London, ON, Canada N6A 5B7
r t i c l e i n f o
rticle history: eceived 31 January 2009 eceived in revised form 5 July 2010 ccepted 5 July 2010 vailable online 15 July 2010
eywords:
a b s t r a c t
The question of the “manner in which an existing software architecture affects requirements decision- making” is considered important in the research community; however, to our knowledge, this issue has not been scientifically explored. We do not know, for example, the characteristics of such architectural effects. This paper describes an exploratory study on this question. Specific types of architectural effects on requirements decisions are identified, as are different aspects of the architecture together with the extent of their effects. This paper gives quantitative measures and qualitative interpretation of the findings. The
oftware architecture equirements engineering mpirical study oftware quality rocess improvement
understanding gained from this study has several implications in the areas of: project planning and risk management, requirements engineering (RE) and software architecture (SA) technology, architecture evolution, tighter integration of RE and SA processes, and middleware in architectures. Furthermore, we describe several new hypotheses that have emerged from this study, that provide grounds for future empirical work. This study involved six RE teams (of university students), whose task was to elicit new
ng a p requ
uantitative and qualitative research rchitecture and requirements technology
requirements for upgradi on a new meta-model for
. Introduction
No one would deny that if we were to extend an existing edi- ce, many of its functional and non-functional features would be f central importance in considering new requirements for the xtension. Yet, in the software engineering (SE) literature, this is ather an understated issue—that is, consideration of existing sys- em design is not a key factor in engineering new requirements.
hile in software practice many developers are indeed aware of he need to assess the fitness of new requirements with the existing ystem design, the approaches are rather subjective and experien- ial. SWEBOK (IEEE SWEBOK, 2004) – the SE body of knowledge – or example, does not describe any practices to deal with this issue. o explore this issue further, we conducted a preliminary survey of 7 professional requirements engineers and software architects. e found that the average rating of the importance of consid-
ring existing system architecture (SA1) when engineering new
equirements was 4.5 (on a 1–5 Likert-scale)—implying that the espondents strongly agreed with this concept. Despite this, sev- ral respondents noted in the qualitative part of the survey that in ctual practice, many organizations neglect this consideration, or
� A preliminary version of this paper was published in Miller et al. (2008). ∗ Corresponding authors.
E-mail addresses: [email protected] (R. Ferrari), [email protected] N.H. Madhavji).
1 For the rest of the paper, the acronym SA refers to system (or software) archi- ecture as a software artefact.
164-1212/$ – see front matter © 2010 Elsevier Inc. All rights reserved. oi:10.1016/j.jss.2010.07.006
re-existing banking software infrastructure. The data collected was based irements decisions, which is a bi-product of this study.
© 2010 Elsevier Inc. All rights reserved.
only perform analysis on existing high-level feature descriptions of the current system, and not the system’s architecture. In many situations, a lack of consideration for an existing system in the new requirements work can lead to rework of requirements and design, incurring extensive costs especially if further downstream in the development process (Boehm and Basili, 2001).
The uptake of this, architecture-requirements, issue in research is not impressive either. It was not until 1994 that the role of an existing SA in requirements engineering (RE) was recognised as important in a panel session. However, at that time, “we still [did] not have a clear understanding of [it]” (Shekaran, 1994a). Shortly thereafter, 5 of the 34 identified indicators of RE success were found to have links with SA (El Emam and Madhavji, 1995). A few years later, the question of an architecture’s role in RE was raised again (Nuseibeh and Easterbrook, 2000). While the awareness of an architecture’s role in the RE process has no doubt increased, to our knowledge, the effects of an existing SA on RE decisions have not been scientifically explored. It is not until such studies are con- ducted, and a dependable body of knowledge created, that practice can begin to use such knowledge in day-to-day projects. As a first step in this direction, this paper describes an exploratory case study
on the effects of an existing SA on RE decisions. Specifically, we ask:
“In which manner does an architecture affect requirements decision-making2?”
2 Decision-making leads from recognition of a problem to be solved to a specifi- cation of that problem or a solution strategy.
2 tems a
p o
c l t s t a t e n s f
m p o a w m s t i a m t w g
n c t r t o a a c
r c r o t c s h s t u o
f p t s h
w
i b
442 J.A. Miller et al. / The Journal of Sys
We explore this question on two fronts: (1) the kind of role a SA lays in requirements decision-making and (2) the specific aspects f the architecture that affect RE decisions.
For point (1) above, it has already been suggested that a SA might onstrain a RE process (Shekaran, 1994b). For example, while ana- ysts could be eliciting requirements to employ a new technology hat requires a specific communication protocol, the current legacy ystem has long implemented a conflicting communications pro- ocol, thereby constraining the current RE strategy. For point (2) bove, while SA aspects are likely largely unique to the domain of hese cases, they would give us an indication of which parts of an xisting software architecture can affect RE decision-making (e.g., on-functional SA areas outside the focus of an RE agent) and, con- equently, which parts of the architecture are critical to document or use by requirements engineers.
Our results indicate that the relationship between SA and RE is ore complex than what is intuitively known in the literature. In
articular, “SA as a constraint” is only one of the four types of effects bserved in our study. The other three types of effects we found re: enabled, influenced and neutral. In short, an enabled effect is here the proposed solution (denoted by the new requirements) is ade feasible because of the implemented decisions in the existing
ystem; an influenced effect is where the architectural configura- ion has an effect on the requirements decision without affecting ts feasibility; and a neutral effect is where there is no noticeable rchitectural effect on the decision. This paper gives quantitative easures on these effects from the study and qualitative interpre-
ation of the findings. Also, in our study, nine architectural aspects ere identified across 117-recorded decisions. Again, this paper
ives quantitative measures and qualitative interpretations. A deeper understanding of the role of SA in RE could open up
ew opportunities for RE and architecting methods, tools and pro- esses. For instance, in the area of planning and risk assessment, he management could make more informed cost estimates of new equirements by considering how the SA has historically affected he various types of requirements. Likewise, in the area of technol- gy improvement, RE and SA tools can be integrated so that analysts nd architects can share, access and change requirements and rchitecture information more easily. We describe several other ases in the paper.
Our empirical study involved six RE teams that gathered new equirements for an existing system and were observed over the ourse of 2 months. The project was in the banking domain and equired the RE teams to elicit and analyse new requirements based n a set of high-level features that needed to be integrated into he current architecture. A requirements decision meta-model was reated as a basis for the development of a requirements-tool that erved to gather data from the participants during the project on ow requirements decisions they were making were affected by pecific aspects of the existing architecture. This paper describes: he study context, participant details, project work involved, the nderlying decision meta-model3 for the data that is gathered, use f tools for gathering data, and the various threats to validity.
The key results are the quantitative characterization of the dif- erent interaction effects mentioned earlier. For example, for this articular system, nine SA aspects affected approximately 60% of he RE decisions. From the findings, we have derived four hypothe-
es that provide a basis for future studies. A general description of ow each of these studies could be conducted is also described.
This paper is structured as follows: Section 2 discusses related ork; Section 3 describes the exploratory study; Section 4 presents
3 The decision meta-model defines the type of data relevant to this study and s a basis for the tool developed for data gathering. The meta-model and tool are i-products of this study.
nd Software 83 (2010) 2441–2455
the results; Section 5 discusses various implications from the results; Section 6 discusses future empirical work and emergent hypotheses from this study, and Section 7 concludes the paper.
2. Related work
This section describes the work that is related to our study. The section focuses on three key aspects: (i) observations, com- mentary and empirical work on the relationship between RE and SA, (ii) technological research spanning RE–SA, and (iii) recent technological-based research on architecture evolution. In Section 2.4, the section concludes with a reflection on the current state of research described in Sections 2.1–2.3.
2.1. RE and SA relationship
There is an increasing interest in exploring and refining the transitions between various activities in the software development process. In particular, the relationship between RE and SA, and their impact on each other was the focus of a couple of workshops 7–9 years ago (STRAW, 2001, 2003). In fact, even earlier, Jackson argued in a panel session (Jackson, 1994) for a tight coupling of the RE and SA processes, suggesting that the most successful developers are those who are able to move relatively more freely between stages within the development cycle. In Kozaczynski (2002), the author discusses that a level of foresight on the part of architects to focus on those requirements that are architecturally relevant can help to mitigate development risk in the software process, by being able to develop the architecture early without all require- ments being elicited. This, early development, can then be fed back to the requirements process to further refine the requirements.
In our earlier work in El Emam and Madhavji (1995), they presented an instrument for measuring RE success. Through an industry field study to design this instrument, we found that in evo- lutionary work, the level of understanding of the existing software architecture can have an impact on the success of the RE process. In understanding the architecture, requirements engineers can pro- vide requirements solutions that are consistent with the current technical and corporate orientation of its organization. In turn, this can lead to better cost/benefit analysis during RE. This early under- standing, however, did not delve into the type of technical effects an existing architecture has on RE decision-making; in this paper, we investigate this issue further.
In Garlan (1994), he recognises that architectural families con- strain system requirements. Further, he identifies that solutions can drive requirements. For example, the architecture of a fam- ily of systems determines the range of variability allowed in a product line. Though not explicitly stated, one can interpret this as not only architectures imposing “constrains” on requirements decision-making, but also as “enabling” and “influencing” such decision-making. This is a central aspect of the current paper.
In Bass et al. (2003), they discuss that different stakeholders of the architecture will have different needs for documentation, and the level of detail provided to them should reflect this. Depend- ing on the stakeholders’ needs, they can be provided with detailed information, some details or overview information of the various architectural views available. The specific architectural aspects that could be important in RE, however, are not mentioned in Bass et al. (2003); our study uncovers these details.
Three previous studies of ours, described below, empiri-
cally examine RE–SA interaction issues from the viewpoints of: architecting-problems rooted in requirements, the effect of using different types of human agents when architecting, and the impact of an SA on requirements characteristics. In Ferrari and Madhavji (2008a), we report on a multiple-case study that investi-
tems a
g a p o w s p o r a t e g a t s r p o c E s d ( g r d a n e w a o n a
2
s t 2 fi u
m t d t w t a p o ( ( r R b p t o t t p f
J.A. Miller et al. / The Journal of Sys
ated requirements-oriented problems that are encountered while rchitecting. Overall, we found that approximately 35% of the roblems encountered during architecting were requirements- riented. Also, specific problem areas together with their severity ere identified (such as, quality satisfaction, requirements under-
tanding and quality drivers) as well as the relative frequency of roblems occurring in these areas. Implications of this work are n improving methods, tools, and techniques to transition from equirements to architecture. In another study, described in Ferrari nd Madhavji (2008b), we report on a controlled-study that inves- igates the impact of software architects having RE knowledge and xperience when performing SA. Specifically, two types of study roups were used, the one type of group had previous training nd/or experience in RE, and the other type of group did not. Both ypes of groups conducted the same architecting project given the ame initial set of requirements from the banking domain. The esults show that the architects with RE knowledge/experience roduced a significantly better architecture (10% difference in the verall architectural quality), and the study also highlighted spe- ific architecting areas where these architects performed better. xamples of these areas include: determining architectural tactics, electing/creating an architectural pattern to satisfy key quality rivers, and interface specification. In a more recent study of ours Miller et al., 2009), we report on a controlled-study that investi- ates the impact an SA has on the characteristics of newly elicited equirements. Two types of study groups were used and con- ucted the same requirements project. One type of group had ccess to a previous SA; whereas, the other type of group did ot. The results showed that a multitude of characteristics (e.g., nd-user focus, technological focus, abstraction, and importance) ere significantly affected by the presence or absence of an SA,
nd the results also showed extent of this effect. Implications f this work are on RE process engineering in the contexts of ew development and legacy systems, and on post-requirements nalysis.
.2. RE–SA technology
There is a growing body of technological work (e.g., methods, oftware tools, processes, development paradigms, notations, etc.) hat is aimed at bridging the areas of RE and SA (STRAW, 2001, 003). The study presented in this paper is meant to elicit new ndings regarding the RE–SA interplay that could then possibly be sed in improving such technologies.
Bass et al.’s stakeholder-centred attribute-driven design (ADD) ethod (Bass et al., 2003) focuses on iteratively building architec-
ures based on the key architectural drivers of the system. These rivers are composed of key requirements and quality scenarios hat shape the architecture. The drivers are input into the process here architectural patterns are created/selected to realize the tac-
ics (i.e., the architectural design choices made) which, in turn, are imed at satisfying the quality scenarios. Tradeoffs emerge in the atterns between various quality attributes, and the architects and ther stakeholders must negotiate a resolution to these tradeoffs similar in principle to the architecture tradeoff analysis method ATAM) (Kazman et al., 2000)) to finalize patterns that would rep- esent an architecture that is most suited to meet the system’s goals. ecently, a prototype tool, called ArchE (Diaz-Pace et al., 2008) has een developed to provide support to the ADD method. This sup- ort is in the form of modelling the functional responsibilities of he architecture, storing the quality scenarios, and through analysis
f the architecture and quality scenarios, the tool suggests tactics hat can be used to satisfy the quality requirements. To date, the ool supports modifiability and performance quality attributes, but rovides plug-in support so users can add reasoning and analysis rameworks for other quality attributes.
nd Software 83 (2010) 2441–2455 2443
In our previous work, we had developed a method that traces architectural concerns back to the requirements—the architecture- centric concern analysis method (ACCA) (Wang et al., 2005). The method uses a concern traceability map (CT-map) that captures and presents architectural design decisions starting from software requirements through to the software architecture and these are then linked to architectural concerns that are identified in the archi- tecture evaluation phase. Through a visual, decision-based, model this method aids in identifying potentially problematic, or sen- sitive, requirements or decisions that resulted in the concerned architectural parts.
Egyed et al., in their component-bus-system and properties (CBSP) method (Egyed et al., 2001), use an intermediate language (and tool support) for expressing requirements in a form that more closely relates to architecture, where requirements are identified and categorized based on various properties such as whether they should be implemented as components, bus, system properties, and so on. This method is focused on early architecting work and is not intended for the entire architecting process. In Hofmeister et al. (2005), the authors deal with the identification and analysis of global factors—those that take into account more holistic issues such as the environment in which the system is built, developing organization, external technological solutions, flexibility or rigid- ity of requirements, and more. Their two-phase method is a means to design and describe a high-level architecture, and analyse and resolve architectural issues introduced by global factors. In partic- ular, the second phase of their approach (global analysis phase), explicitly captures alternative high-level architectural strategies with decomposed design decisions and supporting rationale, and also provides traceability to the requirements.
In Bruin and Vliet (2003), the authors propose an architec- tural design method called quality-driven architecture composition (QAC) where the emphasis is on the reuse of architectural solu- tions. Their method is iterative and starts with the design of an architecture – based only on functional features – and where vari- ability points of the architecture are identified. These variability points are expected to cater to the non-functional requirements. The authors call this initial design the “reference architecture”. Next, the method focuses on the non-functional requirements by iteratively applying known design solutions (i.e., architec- tural and design patterns). The feature-set (FS) graph (which contains pre-existing knowledge about the domain—expressed as requirements) and the resultant design fragments (with their accompanying rationale, assumptions, etc.) that can satisfy the requirements drive this entire process. In Farenhorst et al. (2007), the authors report on a case study that was conducted to explore practitioner’s needs for tool support that focuses on architec- tural knowledge. The study found that practitioner’s require a tool that provides “just-in-time” architectural knowledge, defined as access and delivery of the pertinent architectural knowledge to the right person at any given point in time. Given this broad require- ment, the authors developed an architectural knowledge-sharing portal that stores various types of architectural knowledge and allows for near-instant retrieval through integrated codification techniques.
In Stoll et al. (2008), the authors present the influencing fac- tors method that guides architects in transitioning from high-level stakeholder concerns to preliminary architectural decisions. An “influencing factor” is any stakeholder concern that is considered to play an influential role on the architecture. These influencing fac- tors can be derived from, to name a few, software quality attributes,
business goals, market trends, project experience, etc. The method itself has three main steps: identification of influencing factors, which is accomplished through interviews and workshops with stakeholders; prioritisation of the influencing factors; and lastly, the factors are analysed with respect to their impact on software
2 tems a
q i
i t t m t r n t s r c u o T i a d i
a t m e ( a ( g p p a i
f a s o a
2
e a r c
p a i t a e a ( w a t v a t t a a
444 J.A. Miller et al. / The Journal of Sys
uality attributes, which can then aid the architect to make prelim- nary architectural design decisions.
Cui et al. (2008) present an architectural design approach that s also aimed at transitioning from requirements to architecture hrough the automatic synthesis of candidate architectural solu- ions. The authors construct their approach on a meta-model that
odels issues (architecturally relevant requirements), architec- ural solutions, rationale, and architectural decisions and their elationships. The authors argue that these elements are the key otions for architecture design and the derivation of target archi- ectures. The approach itself has four phases. In the first phase the ystem stakeholders elicit all possible issues (i.e., architecturally elevant requirements). In the second phase, the architects derive andidate architectures for each issue. The third phase involves the se of a formal grammar that facilitates the automatic synthesis f the candidate architectures developed in the previous phase. hese architectural solutions are then presented to the architects n the final phase who can then decide to adopt or reject various spects (or the entire architectures) and provide rationale for their ecision which is then stored for future architectural development
terations. In Schwanke (2005), the author discusses the “good enough
rchitectural requirements process” (GEAR). This process is meant o further refine an initial set of requirements through architectural
eans. The process is based on three architectural requirements ngineering approaches: model-driven requirements engineering where elicited candidate-requirements are modelled as use cases, ctivity diagrams, state charts, etc.), quality attribute scenarios used to elicit, document and prioritise stakeholder concerns), and lobal analysis (a general way of organizing information about the roblem context that surrounds the architecture). The main pur- ose of the process is to show where the above approaches overlap nd where they complement each other, providing insight into the dentification of architectural requirements.
Rapanotti et al. (2004) propose the extension of “problem- rames” into “architecture frames”, which capture information bout architectural styles and their interaction with the problem pace. The benefit of this mechanism is that in introducing solution- riented approaches early in development, one can refine problem nalysis.
.3. Architecture evolution
An area of research that is related to our work is architecture volution, in particular from the viewpoint of methods, processes, nd tools development. In the following section we highlight recent esearch in this area; later in Section 5, we discuss how our study an benefit architectural evolution research.
In Keuler et al. (2008), the authors propose an approach for erforming quality impact analysis on an SA. Their approach uses n aspect-oriented solution to automate integration of automated ntegration of specific concerns (e.g., performance) into architec- ural models, providing specific quality impact evaluations. This pproach is structured in four phases, the first two which can be xecuted concurrently: (1) architectural styles are applied to cre- te an initial style that is specific to the product architecture and 2) quality models for the key quality drivers are created, along ith their accompanying evaluation models. In the third phase,
spects are used to automatically connect the quality models to he existing architecture; and, in the fourth phase, quality specific iews are extracted from the integrated architectural models and
re assessed against the evaluation models from (2). The output of his approach is an identification of the specific parts of the archi- ecture that are affecting the achievement of quality attributes. The rchitect can then use this information for planning changes to the rchitecture as appropriate.
nd Software 83 (2010) 2441–2455
In LaMantia et al. (2008), the authors provide case study results from two architecture evolution projects examined over multi- ple releases where, in each project, architectural modeling was aided by design structure matrices and in accordance with Bald- win and Clark’s design rule theory (Baldwin and Clark, 2000). Design rule theory is a formal theory that explains how design rules (such as splitting or substituting modules) can be used to resolve interdependencies and create modular architectures by specifying the interface between modules. Design structure matrices were designed to support this theory, and are a means of formally model- ing interactions between modules of engineered systems. In short, design structure matrix is a square matrix, in which each mod- ule corresponds both to a row and a column of the matrix. A cell is checked if and only if the design decision corresponding to its row depends on the design decision corresponding to the column. Based on the two case studies results, the authors argue that the use of design structure matrices and design rule theory improved the modifiability of the systems by (1) allowing for different con- current levels of evolution in different modules with no negative consequence on system or development process and (2) facilitated the substitution of risky components with newly proposed com- ponents without substantial change to other parts of the system. The authors conclude that the functionality of design rule theory and design structure can be expanded to provide prescriptive and predictive power in software evolution. Specifically, that the tech- nology could be used to proactively plan for system refactoring.
In Shen and Madhavji (2006), the authors propose a method for developing evolutionary scenarios that provide information con- cerning the impact different types of historical changes (e.g., those related to specific functionality, or those related to external con- cerns such as security, performance, availability, or those due to internal concerns such as maintainability, system defects, etc.) have had on the quality of software architectural elements of interest. Software maintainers, in particular software architects, can use this information when planning system changes. For example, if the maintainer receives a request for a performance modification, they can consult the scenarios to determine the past impact of performance modifications on the modifiability of the system. The scenarios could suggest, for example, that a major refactoring job was required for most past performance modifications, so the main- tainer can then plan accordingly the resource and time allocation to complete the performance modification and any accompanying changes. The evolutionary scenarios focus on the different types of changes that have historically affected a given architectural ele- ment at different times in the evolution of the system. This affect is indicated by measures of the quality of the component, for exam- ple performance, fault-proneness, level of maintainability, etc. The scenarios also provide information on which component sets that have been affected by a given type of change at different times in the evolution of the system. To create these evolutionary scenarios, the evolutionary scenario development method was designed. This struc- tured and automated (where possible) method is needed since the data sources on which the scenarios are constructed can be quite large. The possible inputs to the method can include: bug reports, CVS data, source code, change log fixes, architectural design docu- ments and feature requests. Currently, the method and supporting technology facilitate building evolutionary scenarios that have a focus on maintainability and fault-proneness.
The above work describes research that is focused on performing “off-line” evolution, which basically assumes that the system can be shutdown to perform and integrate the new changes. Other recent
research in the area of architecture evolution proposes technology for performing automated run-time architectural evolution.
In Waignier et al. (2007), the authors propose a framework for performing automated architectural evolution. Specifically, they detail FIESTA, a framework that aids architects in adding
tems a
n f i b t i s i a a t b i l g D a t s s a
2
a t s w
c A E a d 2 h o i i t t a t p w t i c o i w t t f
w c a p o f w i
w
J.A. Miller et al. / The Journal of Sys
ew functionality when performing architectural evolution. Their ramework is generic in that it allows an architecture to be specified n any architectural description language. The framework functions y taking as input a formal specification of the new functionality o be added and the architect then decides where in the exist- ng architecture the new functionality should be integrated. The ystem then automatically makes the transformation into a mod- fied architecture. In another automated architectural evolution pproach, described in Morrison et al. (2007), the authors propose formal architectural description language called Archware-ADL
hat facilitates active architectures, namely an architecture that can e evolved automatically during system run-time based on both
nternal system and external changes. The basic premise of the anguage is to formally model the architecture as part of the on- oing computation, thereby allowing evolution during execution. evelopers can express new components, connectors, constraints nd evolutionary rules in this notation and initiate integration with he system. The system will then accordingly modify and monitor ystem, without any downtime. The authors also propose a set of upport technologies to support this language for these evolution- ry purposes.
.4. Reflection on research
The previous three subsections discuss research in the primary reas that are related to our study: current knowledge pertaining o the relationship between RE and SA; technology aimed at tran- itioning from RE to SA; and, architecture evolution. In this section, e reflect on the current state of research in these areas.
As discussed in Section 2.1, as early as 1994, researchers dis- ussed the importance of the role of an SA in RE (Shekaran, 1994a,b).
few other works have commented on this issue since then (El mam and Madhavji, 1995; Nuseibeh and Easterbrook, 2000), and lso other knowledge-seeking empirical studies have been con- ucted in the area of RE–SA relationship (Ferrari and Madhavji, 008a,b; Miller et al., 2009). However, beyond these works there as been, to our knowledge, sparse research conducted in the area f the role of an SA in RE. When looking at the other direction n the RE–SA relationship, i.e., transitioning from RE to SA, there s an abundance of research work conducted in this area, par- icularly with a focus on technological approaches. In the RE–SA echnological works described in Section 2.2, there is an implicit ssumption that the development is starting from “scratch”, i.e., here is no existing system that is being enhanced. In industrial ractices, however, software development is largely conducted ithin evolutionary processes (IEEE SWEBOK, 2004). Conversely,
he research work presented in Section 2.3 (architecture evolution) s focused on the improvement of the architecting process in the ontext of an evolving system. However, this work solely focused n architecting; the RE process is not explicitly considered dur- ng architectural evolution and is treated as a “black-box” process
here requirements are simply input into the architecture evolu- ionary processes. Therefore, there is little to no consideration in his research for the RE–SA interaction as highlighted in the works rom Sections 2.1 and 2.2.
Thus, there is a need to consider the current system explicitly hen performing RE–SA. Furthermore, there is a lack of empiri-
al evidence regarding the specific interaction effects between RE nd SA. The empirical study presented in this paper is meant to resent detailed quantitative findings on the effect of the presence f a current architecture when performing RE. Such findings can be
ed back into research on state-of-the-art technologies (such as the ork described in Sections 2.2 and 2.3) to facilitate improvement
n RE–SA evolutionary processes. Though the importance of conducting empirical studies in soft-
are engineering (SE) has been recognised (Tichy et al., 1995;
nd Software 83 (2010) 2441–2455 2445
Wieringa and Heerkens, 2006), Shaw’s analysis (Shaw, 2003) of research papers submitted at a prominent 2002 SE conference sug- gests that only 12% were submitted in the category of “Design, evaluation, or analysis of a particular instance” and 0% in the cate- gory of “Feasibility study or exploration”. In Ferrari and Madhavji (2008b), we presented our own analysis of published papers. In the fields of RE and SA, since the year 2000, only approximately 15% of the published papers were in the above-mentioned categories, suggesting that studies such as the work described in this paper are currently rather rare. Our work is meant to help in filling this research gap.
3. The study
Exploratory studies are used when the “research looks for pat- terns, ideas, or hypotheses rather than research that tries to test or confirm hypotheses” (Vogt, 1993). The current research about architectural effects on RE decisions has been anecdotal (Nuseibeh, 2001; Shekaran, 1994a), and thus there is not much grounded theory on this subject. Our study fits the exploratory study char- acteristics. By having multiple cases, we are able to identify trends and patterns beyond a single-case study design.
The following sections deal with: the research questions, par- ticipants, the requirements project, data collection, and threats to validity.
3.1. Research questions
Recall from Section 1 that the intent of this case study was to investigate the role of an architecture in requirements decision- making. We thus have two pertinent research questions:
Q1. How does an architecture affect requirements decision- making?
This question deals with the impact the presence of an archi- tecture has on decision-making in RE. This is accomplished by asking the participants of this study, for every decision that they make, how has the architecture affected that decision. By having a quantitative profile of various architectural effect-types, we can investigate improvement to RE and software architecting technol- ogy with the help of this new information.
Q2. Which aspects of the architecture affect requirements deci- sions?
This second question is intended to probe into the details of Q1. Whereas Q1 was aimed more generally at the effect of architec- ture on requirements decisions, this question aims to characterize the various architectural aspects that are found to have an effect. Through characterization of the different architectural aspects, we can begin to examine improvement opportunities during architect- ing that can optimize future requirements work on a system.
A purposeful tool was developed to gather the data for both the research questions Q1 and Q2 above. The tool is discussed in Section 3.4.2.
3.2. Participants
The population of this study is requirements engineers working in the evolutionary phase (i.e., after the initial release) of a sys-
tem. The participants of the study were 12 graduate and final-year undergraduate level computer science students at the University of Western Ontario who were randomly assigned to 6 teams, each composed of 2 members. The external validity threat from using students in studies is discussed in Section 3.5.1.
2 tems and Software 83 (2010) 2441–2455
3
i t c a F a f s
1
2
3 4
c s s r p
a
3
o a 2 u m p t
e r 8 d s (
g
•
•
•
3
r 2 i w
abstract since it subsumes many possible drivers of change includ- ing (but not limited to) shifting business goals and needs, new contractual requirements, changes in the system’s environment,
8 Some terms to note: Requirements decision—denotes a chosen subset of high- level requirements (or solution strategies) amongst a set of alternatives in order to achieve a goal; Issue—an important topic or problem for debate or discussion relat-
446 J.A. Miller et al. / The Journal of Sys
.3. The RE project
In this study, the participants were given a set of tasks that nvolved upgrading the requirements for an existing banking sys- em as represented by its architecture. Their work involved both reating new requirements and evolving old ones in order to cre- te a new requirements set that satisfied the requested changes. or this purpose, they were given the pre-existing requirements nd architecture documents (described in the following sections) rom the previous version of the system. Each team was given the ame 4 requirements tasks:
. Add Interac service to the existing system. It assumes that the transaction is conducted by the bank’s employee on behalf of the user. For other services, like Internet banking, this time could be different because of external factors like the user’s connection.
. Create a new wireless banking application which would provide features to the customers to carry out basic banking transactions through their cell phones or PDAs.
. Reduce the operational cost of the telephone banking system.
. Increase modifiability in the web banking system.
These tasks were chosen since they constituted a sizeable and omplex RE project that would still be feasible within the con- traints of a University course. We held numerous peer-review essions with a total of six experts to validate these four tasks with espect to their appropriateness in giving a project that met both edagogical and study needs.
The requirements elicitation process and techniques followed re described in Kotonya and Sommerville (1998).
.3.1. The pre-existing requirements document The pre-existing requirements for the system were originally
btained from an external source. These requirements were used to rchitect the previous version of the system (Ferrari and Madhavji, 008b). The final requirements from that project are what were sed as a baseline requirements set for enhancement in the require- ents project on which the study was conducted. Thus, the study
roject essentially involved one iteration of an evolutionary cycle of he system’s requirements.
However, these requirements were re-validated by several xperts for acceptability in the enhancement project (i.e., the four equirements tasks described earlier). There were approximately 0 requirements in the set, and supporting use cases and sequence iagrams for ten of the key functions of the system. The document tructure followed the guidelines from Kotonya and Sommerville 1998).
We list here a few example requirements in natural language to ive their flavour:
The system must complete a transaction in less than 3 s. The transaction is conducted by the bank’s employee on behalf of the user. For other services, like Internet banking, this time could be different because of external factors like the user’s connection. A customer shall be able to deposit money using ATM into the indicated account by cheque or cash. A customer shall be provided access to Internet banking services based on valid bank account number, user defined password, and access permissions set out for the bank customer.
.3.2. The architectural document
The architectural document given to each of the RE teams
esulted from the described previous study (Ferrari and Madhavji, 008b). That study involved a set of 16 software architecting teams
n an academic setting, each of which worked to create a soft- are architecture, using the ADD method (Bass et al., 2003). The
Fig. 1. A meta-model for RE decisions.8
participants in that study created their architectures based on the requirements mentioned in Section 3.3.1. That study also involved identifying one particular architectural document as being of the highest quality based on an instrument designed for this purpose (Ferrari and Madhavji, 2008b).
The architecture in question was documented in a 161-page document and included information on: quality attribute scenarios, tactics employed, module decomposition views, user/layer views, class views, component and connector views, deployment views, interface specification, work assignment view, sequence diagrams, state charts, and architectural rationale.
3.4. Data collection
In order to gather appropriate data to answer the two research questions, Q1 and Q2 (see Section 3.1), we first designed a meta- model for requirements decisions. Also, to simplify data collection and organization, we developed a software tool based on this meta- model. Furthermore, we had specific measures in place to ensure that quality data would be obtained from this study. These issues are described below in more detail.
3.4.1. The decision meta-model The decision meta-model specifies the types of entities and
relationships involved in the myriad of decisions underlying the requirements process. This meta-model, therefore, can guide data gathering. Since research on requirements decisions is limited, there was no established meta-model available which fitted the specific investigative needs of this study. Instead, a combination of elements from two different sources was used: Ramesh and Jarke’s rationale submodel (Ramesh and Jarke, 2001) and Wang and Mad- havji’s traceability meta-model (Wang et al., 2005). The integrated meta-model is illustrated in Fig. 1 and uses UML notation to depict the elements and links.
The meta-model captures the key notions of decisions, assump- tions, requirements and solution approaches. It links various elements to the system through decisions. The input to the model is the change driver element. Change driver is left intentionally
ing to the acceptance/rejection of a solution approach; System—computing system of interest; Rationale—why a requirement is needed with respect to the goals it real- izes; Argument—statement supporting or refuting the solution approach; Domain knowledge—the valid knowledge used to refer to an area of human endeavour (in this case the Banking domain); Assumption—a statement which is considered true regarding any aspect of system development.
J.A. Miller et al. / The Journal of Systems a
a t
m a i t e t ( a a r
c t r a t a a u s
r a E r r a t d
( i d
t
Fig. 2. A sample decision tree from the meta-model.
nd end-user change requests. In our study, the change drivers were he four project tasks given to the teams (see Section 3.3).
One of the primary attributes that differentiates our meta- odel from the earlier ones (Ramesh and Jarke, 2001; Wang et
l., 2005) is that, in our model, requirements decisions relate only ndirectly to requirements, issues and assumptions, through solu- ion approaches. That is, in the ensuing instance-level model (or nactment of the model), each solution approach (i.e., a strategy o meet high-level requirements) involves its own set of issues e.g., cost implications, constraints, actions, etc.), requirements and ssumptions (see Fig. 2). These are only instantiated if the solution pproach is accepted through a decision (and hence the “indirect” elationship).
For example, a decision concerning the reduction of operating osts in the telephone banking system might involve two solu- ion approaches; reducing the number of human operators and/or educing the available functionality of the system. Both solution pproaches are feasible, and the RE team must choose (based on he associated issues) whether or not to implement4 either of the pproaches. Note that it is possible to choose both or neither. Once decision is made, rationale can be given describing why a partic- lar decision was made (e.g., why a particular solution approach hould be implemented over another solution approach).
Specific issues can apply to many solution approaches. Each equirement and assumption is associated to a single solution pproach, which can then be traced to one or more decisions. ach requirement has its own rationale, underlying assumptions, elationships to other requirements, importance and other project- elated attributes such as cost estimate and tasks. However, these
re not elaborated in the meta-model for simplicity. It is around his model5 that the data-collection tool (see Section 3.4.2) was esigned.
4 Note: Though the RE team is not expected to do downstream development work design, coding, testing, etc.), it is evident here that their decision here is carving an mplementation path through the “solution approach” they would choose, thereby enoting a problem–solution space relationship. 5 Prior to the start of the study, peer review with RE experts was used to validate
he model’s quality.
nd Software 83 (2010) 2441–2455 2447
For each of the elements that help to make up the meta-model, relevant information was captured by the tool. Each element had a unique set of attributes that were captured. The attribute that is of particular interest here is the role that the architecture played in requirements decision-making. The role of the architecture is denoted by whether it acted as an effect (constrained, enabled, influenced, or none) on the requirements decision, and the aspect of the architecture that had the effect. The system and domain knowl- edge elements are not directly implemented in the tool, but are meant to provide a context for the rest of the elements and how they fit with the overall system.
3.4.2. Data-gathering tool The data-gathering tool could best be described as a decision-
centric requirements engineering tool. The subjects logged each decision they made into the tool. Each decision had a series of potential solution approaches associated with it, all of which were also logged (see Fig. 3 for an example screenshot). Underlying the decisions captured, and the way the tool operated is the deci- sion “meta-model” described earlier (see Fig. 1). The tool was implemented in Visual Basic 6 (VB6). It had the dual purpose of sup- porting the subjects’ work and of recording decision data relevant to this study.
Because of this semi-automated tool, data quality could be ensured in several ways that a manual tool (such as forms that subjects must fill out) could not. For example, the subjects could be required to fill in essential fields at the right time such as a require- ment’s rationale when a requirement is logged so that there’s no danger that they might be left blank or, worse, filled in at a later time when the knowledge is no longer fresh. Other fields (e.g., the time of modification) could be generated automatically, thus, alle- viating the subjects’ workload while guaranteeing correctness of the data.
3.4.3. Data collection The data-collection phase of this study took place over a span
of 2 months. To help ensure the quality of the data, each team was given 1 h a week to meet with a system “stakeholder” played by the course’s teaching assistant. During the meetings the sub- jects had the opportunity to ask questions about the company’s needs regarding the new system. Their work to date was reviewed priori to, and discussed at, these meetings to ensure that the sub- jects properly understood how to use the tool for logging data. Additionally, e-mail communication was used to answer questions regarding tool usage.
3.5. Threats to validity
Based on Johnson and Christensan (2004), three types of threats that might apply to the type of study conducted here were iden- tified: external, construct, and conclusion validity. Because we are not attempting to demonstrate causality between variables, threats to internal validity are not a concern.
3.5.1. Threats to external validity External validity refers to the degree to which the results
of a study can be generalized across a population (Johnson and Christensan, 2004). Threats to external validity occur when researchers draw incorrect conclusions about the population based on the sample data (Creswell, 2003).
Population validity is the ability to generalize the study results
from the sample to the population. Because exploratory studies on students have become so prevalent, there is much work done to explore the population validity of students. Specifically regarding SE related student-based studies in academic settings, important results have been found in several cases, e.g., in requirements triage
2448 J.A. Miller et al. / The Journal of Systems and Software 83 (2010) 2441–2455
Fig. 3. A sample screen shot from the decisions data-gathering tool. The left-side pane lists the decisions that have been logged in the system. The highlighted decision is the one being currently worked on. The right-side of the screen is split between two sets of windows: the left-side is where architectural constraint information is logged, and the right-side is where architecture acting as an enabler information is logged.
( t t a c e s i s
3
c p s F o m w r t c
Runeson, 2003), code inspection (Carver et al., 2002), and in lead- ime impact assessment (Host et al., 2000). We do acknowledge the hreat in generalizing to experienced requirements engineers and rchitects; however, there is no evidence suggesting that the results ould not be generalizable to, at the very least, novice requirements ngineers and architects in industry. Regardless, exploratory studies uch as this are an important first step towards eventually solidify- ng a body of knowledge and providing the groundwork for future tudies in wider contexts.
.5.2. Threats to construct validity Construct validity refers to the extent to which a measurement
orresponds to theoretical concepts (constructs) concerning the henomenon under study. In this study, the constructs (e.g., deci- ions) were operationalized through the decision meta-model (see ig. 1) and the tool that was built on this model. We held numer- us peer-review sessions with a total of six experts to validate the
eta-model and tool with respect to the theoretical constructs we anted to investigate (see Section 3.4.1). Also, at no stage in the
esearch process did we come across any instant of data or rela- ionship that questioned the validity of the meta-model or the tool’s apability in capturing data pertaining to the meta-model. We are
thus confident in the effectiveness of these artefacts for collecting data pertaining to the study’s constructs.
3.5.3. Threats to conclusion validity Conclusion validity is the degree to which conclusions we make
based on our findings are reasonable (Trochim, 2006). There are two accepted principles for improving conclusion validity (Trochim, 2006) that applied to our study: ensuring reliability of data mea- surements and proper implementation of study processes. For reliability of data measurements, we utilized a data-collection tool and weekly meetings to ensure tool was utilized correctly (see Sec- tion 3.4.3). Proper implementation was in-place by having a single researcher involved in the study design to perform the various research tasks. Additionally, we discuss the conclusions in the last section of the paper, and there we demonstrate that all our con- clusions are rooted in the results, thereby maintaining conclusion validity.
4. Results
This section discusses the findings of the study. We describe first the manner in which the architecture affects requirements deci-
tems and Software 83 (2010) 2441–2455 2449
s t d
4 (
s ( w o
4
m e e b a a t f e k
E
s
w
s
Q a r
w b r w c t
E
f
b
U
p
d
E
o t
t b s
f ar
ch it
ec tu
ra l
im p
ac t
o n
re q
u ir
em en
ts d
ec is
io n
s.
A rc
h it
ec tu
ra l
A sp
ec ts
D ec
is io
n s
E x
is ti
n g
h ar
d w
ar e
N F
ch ar
ac te
ri st
ic s
(s am
e su
b -s
y st
em )
N F
ch ar
ac te
ri st
ic s
(d if
fe re
n t
su b
-s y
st em
)
R eu
sa b
le m
o d
u le
s A
rc h
it ec
tu ra
l p
at te
rn s
M o
d ifi
ab il
it y
St ru
ct u
ra l
fe at
u re
D ec
is io
n al
re ad
y m
ad e
C o
m m
u n
ic at
io n
N u
m b
er o
f d
ec is
io n
s af
fe ct
ed
2 4
1 6
5 3
5 1
4 3
3 6
2 4
8 5
3 3
4 4
6 3
1 0
2 2
0 3
0 0
0 2
7 4
8
si o n
s: 1
1 7
To ta
l #
o f
“e ff
ec t-
co u
n ts
”: 1
2 2
n d
ec is
io n
ca n
b e
af fe
ct ed
b y
m o
re th
an o
n e
ar ch
it ec
tu ra
l as
p ec
t an
d th
er ef
o re
th e
n u
m b
er o
f ef
fe ct
-c o
u n
ts m
ay n
o t
eq u
al th
e n
u m
b er
o f
af fe
ct ed
d ec
is io
n s.
J.A. Miller et al. / The Journal of Sys
ions (Q1). Then, we describe the quantitative findings related to he specific architectural aspects that affected the requirements ecisions (Q2).
.1. How an architecture affects requirements decision-making Q1)
The six project teams recorded a total of 117 requirements deci- ions, all of which related to the four requirements tasks assigned described in Section 3.3). A significant portion of these decisions as affected in some way by the architecture. We describe the types
f effects found in our study and their characteristics.
.1.1. Types of architectural effects We identified four types of architectural effects on require-
ents decisions from our data, shown in Table 1 (leftmost column): nabled, constrained, influenced, and neutral. An architectural ffect is of type enabled if it makes a solution approach (more) feasi- le because of the current architectural configuration. Conversely, n architectural effect is of type constrained if it makes a solution pproach less (or in-) feasible. An influenced is where the architec- ural effect altered a requirements decision without affecting the easibility of its solution approaches. Finally, the neutral type of ffect is one where there is no noticeable architectural effect of any ind.
xample 1. Enabled
Decision: Implement a back up system for the Interac banking ystem.
Solution approach 1 (rejected): Introduce new web server which ill be used as a backup for Interac transactions.
Solution approach 2 (accepted): Use Internet sub-system web erver as a backup for Interac transactions.
Architectural enabler: The web server for Internet already exists. ueue will allow us to hold over 500 transactions and deal with ll the requested transactions. Overall system requirement 1.19 equires that the system should not fail in case of overload.
In this example the decision to use the existing Internet banking eb server as a backup for the Interac sub-system was made easier
ecause the team in question knew that existing performance and eliability requirements were sufficient to accommodate the extra orkload. This is an example of being enabled by “non-functional
haracteristics from a different sub-system” (an aspect of the archi- ecture) than the one being worked upon.
xample 2. Constrained
Decision: Establish communications protocol(s) that will be used or the wireless banking system.
Solution approach 1 (accepted): Communication protocol should e GPRS (general packet radio service).
Solution approach 2 (rejected): Communication protocol shall be MTS (universal mobile telecommunications system).
Architectural constraint: Architecture document clearly stated a reference for GPRS; otherwise we would have chosen UMTS.
Here, the example is of a decision being constrained because the ecision had already been made in the architectural document.
xample 3. Influenced
Decision: Deploy the Interac system. Solution approach 1 (accepted): Develop the Interac system based
n the conceptual model of system based on the current implemen-
ation of the ATM sub-system.
Architectural influence: The architecture document defines func- ionality for the ATM system. The Interac system can be loosely ased upon this conceptual model since the ATM system has been uccessfully implemented and maintained. Therefore, the presence Ta
b le
1 C
h ar
ac te
ri st
ic s
o
T y
p e
o f
ef fe
ct
E n
ab le
d C
o n
st ra
in ed
In fl
u en
ce d
N eu
tr al
To ta
l #
o f
d ec
i
N o
te th
at a
g iv
e
2 tems a
o s
e p s n p
E
m
s
l a
g
n c m o
e t c w
i r I t n r a s e b
4
t b c h b t 5 6 a c a o i w i i
e t
t
450 J.A. Miller et al. / The Journal of Sys
f the ATM sub-system and how it was implemented influences the olution approach for the Interac system.
This is an example where the decision was not constrained or nabled; nothing about the ATM sub-system makes any of the pro- osed solution approaches more or less feasible. However, for the ake of consistency, the requirement engineers chose to model the ew Interac system after the ATM system. This decision is an exam- le of a decision being influenced by architectural patterns.
xample 4. Neutral
Decision: Determine support for different languages in the obile banking application. Solution approach 1 (rejected): English will be the only language
upported. Solution approach 2 (accepted): English language as the default
anguage of the system, with other languages to be downloaded nd installed on request.
Solution approach 3 (rejected): Provide support for many lan- uages together with the application.
Architecture effect (none): The mobile banking application has ot been developed, and therefore the technical challenges asso- iated with implementing language support on a wide-variety of obile devices are considered outside the scope of the current
verall system architecture. Example 4 demonstrates a decision that was unrelated to the
xisting architecture. In this situation, the mobile banking applica- ion has not yet been implemented so the requirement engineers an consider various solution approaches for language support ithout considering the current architecture.
Note that the described effects are “technical” in nature. That s, our focus is on the “architectural basis” for deciding whether a equirement decision is enabled, constrained, influenced or neutral. n a given software project, there are other factors that also need o be considered in prioritising requirements and in release plan- ing, e.g., implementation cost, revenue potential, and resource equirements. Irrespective of these factors, it is invaluable to know t elicitation-time what the architectural effects are on the deci- ions being made. Thus, for example, with revenue potential being qual among two competitive decisions, an enabled decision would e more favourable than a constrained one.
.1.2. Architectural impact characteristics Of the 117 requirements decisions mentioned in Table 1, 69 of
he decisions were affected by the architecture. A decision could e affected by more than one architectural effect (for example, the hoice of upgrading a database could be enabled by the current ardware configuration, but also be constrained by poor modifia- ility in the system components that would need to interact with he database). With reference to Table 1, in our study there were such cases, so we had a total of 122 “effect-counts”.6 Out of the
9 affected decisions, an effect-count of 74 out of 122 (61%) were ffected by the architecture. This is a substantial number of effect- ounts that were affected in some way. There is, more or less, n even-split between those “enabled” and “constrained”, which utnumber the category of “influenced” by a factor of 5. Equally mportant is to note that 48 (41%) of the requirements decisions
ere not affected by the architecture (i.e., type neutral). Also, all nstances of architectural effects on requirements decision-making
n our study fit into the defined types of effects.
In previous literature (Shekaran, 1994b), only the “constraint” ffect-type was identified. In Section 4.1.1, we identify additional ypes of effects. Also, in this section, we give quantitative character-
6 The effect-count includes some decisions in more than one category of effects, hus the summation does not tally, or the % is more than 100.
nd Software 83 (2010) 2441–2455
istics of the various effect-types. However, it should be noted that different application systems are expected to have different quanti- tative values because these values depend on factors specific to the development of individual products or systems. Still, it is a subject for future studies as to whether there are approximate quantita- tive ranges for different effect-types across different applications and application-domains.
4.2. Architectural aspects affecting requirements decisions (Q2)
The types and quantitative characterization (see Table 1) of architectural effects on requirements decision-making (Q1) is complemented by the findings of the different aspects of the archi- tecture that had impact on requirements decisions (Q2).
Table 1 shows, on the top, 9 architectural aspects that were found to affect requirements decisions in the project. These aspects are:
1. Existing hardware: Decisions that were affected by the existing hardware in the system.
2. Non-functional characteristics (from the same sub-system): Deci- sions that were affected by non-functional characteristics of the same sub-system as the one with which the decision was con- cerned.
3. Non-functional characteristics (from a different sub-system): Deci- sions that were affected by non-functional characteristics from a different sub-system than the one with which the decision was concerned.
4. Reusability of modules: Decisions that were affected by the pos- sibility of reusing existing modules.
5. Architectural patterns: Decisions that were affected by the choice of architectural patterns already implemented.
6. Modifiability: Decisions affected by existing features that were known to be easily modifiable.
7. Structural features: Decisions that were affected by structural features of the existing SA.
8. Decisions already made: Decisions that were affected when it was realized that the decision in question had already been made in the existing architecture.
9. Communications: Decisions that were affected by the existing choice of communications protocols.
Below, we analyse architectural aspects against effect-types and against the project groups.
4.2.1. Architectural aspects across effect-types Table 1 depicts the role of the architectural aspects (top row) in
relation to the type of effects (leftmost column) on the total set of “effect-counts” (122) recorded by the project teams.
Though the category influenced occurred less frequently than enabled and constrained, they are still noteworthy. In our study, influenced usually denoted that solution approach used in another part of the system was being used to solve the problem at hand. For example, an architectural pattern might be chosen because it has been implemented successfully elsewhere in the system.
While this may suggest a movement towards a more homogonous architecture, an aspect acting as an influence on future RE decisions may be less foreseeable (by a software architect) than those acting as types enabled and constrained. In particular, whereas enabled and constrained are related to creating requirements which are consistent with the established architecture and previously
made decisions, influenced involve implementing previous (or sim- ilar) decisions in a new context (i.e., a different part of the system than was originally intended). The risk associated with this, how- ever, is not clear. Thus, if an aspect is known to be of type influenced, the architect should be aware that design decisions involving that
tems and Software 83 (2010) 2441–2455 2451
a n t
4
a t i (
b T ( m
i a s p m w a t 5 a u e
s r t b b s 5
5
f
5
t a r m c t t ( F m b T b p o r t b l b
b et
w ee
n ar
ch it
ec tu
ra l
as p
ec ts
an d
p ro
je ct
te am
s.
A rc
h it
ec tu
ra l
as p
ec ts
D ec
is io
n s
E x
is ti
n g
h ar
d w
ar e
N F
ch ar
ac te
ri st
ic s
(s am
e su
b -s
y st
em )
N F
ch ar
ac te
ri st
ic s
(d if
fe re
n t
su b
-s y
st em
)
R eu
sa b
le m
o d
u le
s A
rc h
it ec
tu ra
l p
at te
rn s
M o
d ifi
ab il
it y
St ru
ct u
ra l
fe at
u re
D ec
is io
n al
re ad
y m
ad e
C o
m m
u n
ic at
io n
# A
ff ec
te d
# T
o ta
l %
A ff
ec te
d
0 0
0 0
0 2
0 0
2 4
1 2
3 3
1 0
0 0
0 1
1 0
1 4
1 2
3 3
0 0
3 1
2 0
0 0
0 6
1 6
3 8
2 3
1 0
1 4
2 3
3 4
3 2
3 8
8 4
0 1
7 4
0 1
0 3
0 1
6 2
4 6
7 0
5 0
0 0
0 0
0 2
7 1
5 4
7
3 9
2 0
6 6
6 4
6 9
6 9
1 1
7 5
9 (1
1 7
) 3
8 1
7 5
5 5
3 5
8 ed
(6 9
) 4
1 3
2 9
9 9
9 6
9 1
3
J.A. Miller et al. / The Journal of Sys
spect may have ramifications in other parts of the system that may ot be obvious. Care should therefore be taken when architecting hese aspects.
.2.2. Architectural aspects across project groups Table 2 shows the number of requirements decisions that were
ffected by each architectural aspect and for each of the six project eams. The table shows that the architectural aspect “NF character- stics of a different sub-system” affected most number of decisions 20; or 17% of 117 decisions; or 29% of 69 affected decisions).
Besides this, all the remaining architectural aspects affected etween 4% and 13% of the affected decisions (see last row in able 2). Also, we see that in Table 1, the aspect “NF characteristics different sub-system)” has the greatest % of “enabled” require-
ents decisions (16 of 36, or 44%). We do see some discrepancies, however. While “NF character-
stics (different sub-systems)” was the most active architectural spect (see Table 2), the instances of affected requirements deci- ions came from groups 3, 4 and 5. One explanation for this henomenon could be that this particular aspect depended on how uch effort the subjects put into understanding sub-systems that ere non-local to those in their focus of attention. Indeed, while
cquiring an understanding of the other sub-systems in the archi- ecture did actually affect the decision-making of groups 3, 4 and , it is possible that the other groups simply did not focus their ttention on seemingly unrelated sections of the architectural doc- mentation. We do not have data for this analysis, and so future mpirical studies could help explain this phenomenon.
Despite this variance between teams, we include all data points ince this a multiple-case study. However, including this data esults in 59% of requirements decisions being affected by archi- ectural aspects (as seen in Table 2, last column, 3rd row from the ottom), while their exclusion would result in 52% of the decisions eing affected, so there is not much difference. Thus, we will simply tate that the architectural aspects listed affected approximately 0% of the requirements decisions.
. Implications
There are a number of implications for SA and RE of the findings rom our study.
.1. Planning and risk management
The analysis and categorisation, during the RE process, of archi- ectural effects on RE decisions (see Section 4.1.1) could help rchitects to separate the more easily implementable, enabled, equirements from the more difficult to implement (or compro- ised), constrained, requirements. This separation of concerns
ould be useful from the point of view of project planning (e.g., ime-to-implement, resource allocation, requirements prioritisa- ion and scheduling), risk management (e.g., implementability) Boehm, 1988), and product evolution (e.g., new feature planning). or example, one group in the dataset elicited high-level require- ents to reduce the cost of telephone operators in telephone
anking by introducing an automatic speech recognition system. hese requirements were “enabled” in two principle ways: one, y readily available COTS systems/components from the market- lace and two, by the modifiability of the current implementation f the telephone sub-system. The same group elicited high-level
equirements for the mobile banking application, specifically that he existing infrastructure (i.e., servers and their throughput) could e used to handle the mobile banking application transaction
oad. However, these requirements were assessed as “constrained” ecause of the existing performance demands from the other major Ta
b le
2 T
h e
re la
ti o
n sh
ip
T ea
m
1 2 3 4 5 6 # T
o ta
l %
o u
t o
f to
ta l
% o
u t
o f
af fe
ct
2 tems a
t n S t p t t a s
5
c m ( A i q p w i i q m
a p s c i d t
5
c p a i i a n o u u t i i S c q e o a a e
t t
452 J.A. Miller et al. / The Journal of Sys
ypes of access to the system (e.g., Internet, teller, etc.). So, for plan- ing purposes, the management had to decide: should I upgrade the A in order to implement the requirements for the mobile applica- ion, which has a potentially high positive impact on the customer’s oint of view? Or, should I implement instead the requirements for he automated phone system, where these are less desirable from he customer point of view but less-time consuming to implement nd hence can lead to releasing the system faster and thus start aving money by removing the human telephone operators?
.2. RE and SA technology
Similarly, this separation of concerns of architectural effects ould help researchers and tool developers to enrich the require- ents elicitation and analysis tools (e.g., DOORS, Requisite pro, i*
Liu and Yu, 2001), etc.) which, in turn, could enrich SA tools (e.g., rchE (Diaz-Pace et al., 2008), Software Architect, etc.) in mak-
ng judicious choices of architectural tactics and patterns to satisfy uality requirements. Currently, RE and SA tools do not consider the resence of an existing system when performing further RE and SA ork, and therefore do not facilitate the presentation or analysis of
nformation describing the RE–SA interaction effects (such as the nformation in Table 1). Integrating this information, and subse- uent analysis support, into RE–SA tools could then enable users to ake decisions based on information that is currently left implicit. Likewise, this separation of concerns can help in implementing,
utomatic, dialogue-triggering mechanisms in RE-to-SA workflow rocesses (Georgakopoulos et al., 1995), especially for the “con- traint” category of requirements. That is, the RE and SA agents an be notified automatically to resolve the tradeoffs between mplementing a constrained decision (at the expense of customer issatisfaction) and implementing an unconstrained decision (at he expense of architectural modifications).
.3. Architectural evolution
Historical trends of aggregate quantitative data (as in Table 1) an aid in SA management and in opportunistic or restrained RE ractice. For example, if the trend shows that too many RE decisions re constrained by the specific aspects of the legacy SA (e.g., 8, 5 or 6 n the “constrained” row in Table 1) then this might call for: (i) exam- nation of SA practices and developing checklists to ensure that rchitects are not inadvertently restricting potential future busi- ess goals; (ii) restructuring7 the SA to align it with business goals; r (iii) restraining the RE process (from attempting to integrate nconstrained requirements into the constraining parts of the SA) ntil such time that the architecture has been adequately restruc- ured. Conversely, trends of too many enabled decisions (e.g., 16 n the “enabled” row for “NF characteristics (different sub-system)” n Table 1) could possibly indicate that the enabling aspects of the A are, at least, technologically supportive of the new ventures and an unleash RE to be more opportunistic. This type of analysis and uestioning is not a part of architecting methods (e.g., ADD (Bass t al., 2003), GRL (Liu and Yu, 2001), and CBSP (Egyed et al., 2001)) r architecture evolution approaches (e.g., ArchWare (Morrison et
l., 2007), ESDM (Shen and Madhavji, 2006), FIESTA (Waignier et l., 2007)), and, doing so, could allow for improved architectural volution support.
7 SA restructuring can include such tasks as: capability analysis (of the SA as o whether it can cope with stakeholder scenarios), tactics and pattern choices, echnology assessment, deployment strategies, and others.
nd Software 83 (2010) 2441–2455
5.4. Tighter SA–RE integration
With over 50% of the RE decisions being affected by an SA (see Table 2), and many of these (29% or 20/69) originating from the aspect “NF characteristics of a different sub-system”, this is strong empirical evidence in favour of integrating software architect- ing and RE processes more tightly (Nuseibeh, 2001). Specifically, the SA agents could work with the RE agents during require- ments elicitation, negotiation and feasibility analysis in order to provide critical insight on the technical feasibility of the elicited requirements in terms of them being constrained or enabled from a different sub-system as opposed to the sub-system they are working on.
Therefore, a hypothesis emerges (see Section 6) that, in order to reduce the amount of backtracking and requirements-rework (and also reduce the associated project costs), it is important that the architects provide “live” feedback to the RE agents on these potential system-wide “constraints” and “enablers”.
However, due to resource constraints in RE–SA processes of a software project (for example, in some projects it may not be possible for requirements engineers to have extensive interaction with the architects), at the very least requirements engineers could analyse different sub-systems than the one they are working on to possibly discover more local requirements decisions that could be enabled. If this is so, requirements engineers could be trained, and appropriate tools developed, specifically for this circumspec- tive analysis in order to yield more enabled solutions for better service and satisfaction to the end-user. As mentioned in Section 1 of this paper, the current industry practice does not align with this recommendation.
5.5. RE-to-SA feed-forward process
Iterative development approaches (such as RUP (Kruchten, 2001) and spiral (Boehm, 1988)) tend to promote that significant chunks of requirements are validated and prioritised preceding the development effort. While this may be quite appropriate in many situations, there is room to be agile in some situations across RE-architecting processes by introducing “feed-forward” processes from RE to SA. In particular, requirements engineers can pack- age critical information and deliver this to the architects prior to the delivery of the validated new requirements. For example, in our case study projects the requirements engineers could have packaged information about the four architectural categories of high impact (see Section 4.1.2: existing hardware, NF standards (same sub-system), NF characteristics (different sub-system), and architectural patterns), the specific requirement decisions that are affected, and how they were affected (e.g., constrained, enabled or influenced). This package of information, if made available to the architects “ahead of time”, could facilitate groundwork for spe- cific architectural enhancements, and change, while the rest of new requirements are still being elicited in the RE process. We note that agile practices (Larman, 2003) do not explicitly promote such feed-forward processes from user stories to system develop- ment.
5.6. Increased middleware
The neutral type of effect has a significant amount of cases (approx. 40%, see Table 1). Neutral cases actually mean that the developers will likely have to “wire in” the design and code for
a new requirement into the system much more deeply than in the “enabled” cases where, for example, the groundwork would already have been prepared in the existing architecture for the new requirements to be implemented. Deeper the “wiring in”, higher the software costs in general and more arduous the development.
tems a
T t d
5
m s i i
6
p a t t o f a o s q p h g
d t
H a a
o s g a e g w m t g r v r
H s f
s m i t i i
a t t N g
J.A. Miller et al. / The Journal of Sys
hus, some of the “wiring-in” work could possibly be reduced in he future by increased “middleware” strategy in the architectural esign.
.7. Analysis
So, as we see above, there are quite a few implications of deter- ining architectural effects on requirements decisions: on early
oftware development practices, methods and tools. The identified mplications are threads for further empirical work to ground them n development processes.
. Future empirical work
One purpose of an exploratory study is to lay a foundation for ossible future work on the theme of the research so as to build an ppropriate body of knowledge (IEEE SWEBOK, 2004). In a sense, he exploratory study is conducted in a “bottom-up” manner, where he research question acts as a guide to collecting a wide range f data about the research topic, and the findings are discovered rom the exploratory analysis of this data. In an effort to lay such foundation, it is important to identify any emergent hypotheses r investigative questions from this research. From such hypothe- es, it would then be possible to conduct, in a “top-down” manner, uantitative studies that focus on specific research issues. The main urpose of conducting a “top-down” study is to statistically test the ypothesis to lend quantitative support to the topic being investi- ated.
From the results of our study and their implications, below we escribe the following four emergent hypotheses that could be ested in future studies and how they could be tested:
ypothesis 1. If the architects provide “live” feedback to the RE gents on potential system-wide constraints and enablers, then the mount of requirements-rework will be reduced.
See Section 5.4 for a more detailed discussion of the background f this hypothesis. To test this hypothesis, we would need to mea- ure the amount of requirements rework between two different roups of software engineers. This measurement could include the mount of requirements-rework needed to be done, and also the xtent of the rework (i.e., effort and time). One of these study roups would have requirements engineers and architects who are orking together in an integrated manner to develop the require- ents and architecture; the other type of group would not have
he requirements engineers and architects working as closely inte- rated. For this hypothesis, the independent variable would be the equirements and architecting process used, and the dependent ariable is the time and effort expended performing requirements- ework.
ypothesis 2. Non-functional (NF) characteristics of a non-local ub-system significantly affect (enable or constrain) requirements or the local sub-system being worked on.
In Table 1, we see that NF characteristics of a different sub- ystem than the one being worked upon affected requirements ore than any other aspect. This could have potentially important
mplications on RE–SA technology as discussed in Section 5. Despite his importance, prior to investigating into new technologies, there s a need to replicate this study in different domains and contexts n order to determine generalizability.
To test this hypothesis, therefore, two types of RE–SA groups
re needed for the study: one that is given the entire architec- ure including information regarding the NF characteristics of all he sub-systems; whereas, the other group does not receive this F information. Both groups would elicit requirements for a sin- le sub-system and, as in this study, architectural aspect analysis is
nd Software 83 (2010) 2441–2455 2453
performed and the number of impacted requirements is logged and statistically compared. The independent variable would then be the presence/absence of NF characteristic information of non-local sub- systems, and the dependent variable would be the reported number of impacted requirements.
Hypothesis 3. If the history of interaction effects between SA and RE is used effectively, then the time/effort spent performing evolutionary work in requirements and architecture processes will decrease.
As discussed in Section 5, maintaining and using the history of information presented in Table 1 could be useful for evolutionary work in the requirements and architecting processes. This hypothe- sis aims at providing scientific evidence as to whether or not having such information is useful and, if so, to what extent.
A controlled experiment involving two study groups could be used to test this hypothesis. Development teams expected to enhance a system (both requirements and architecture) would be used. One type of study group would be given the histori- cal interaction effect information from the past revisions of the system; whereas, the other group would not receive this informa- tion. Process data such as effort and time would be gathered and then analysed to determine any statistically significant differences between the two types of groups. The independent variable is the historical information, and the dependent variable is the time and effort spent in performing an evolutionary phase in an RE and SA project.
Hypothesis 4. Architectural communication protocols used in the current system have a significant effect on new requirements.
In Table 1 in Section 4.2, communication protocols used in the architecture have an effect on new requirements. Despite the finding that the effect is mostly constrained, this is more likely due to a function of our product circumstances, thus we gen- eralize this hypothesis for all the types of effects. Establishing further evidence of this claim can lead to improved RE and SA technology where this issue is more explicitly considered in those processes.
To test this hypothesis, a study with two types of RE groups enhancing the requirements for an existing system is needed. One type of group will be given an existing system architecture with fully realized communication protocols. The other type of group would be given an existing architecture, however, the com- munication protocols would be undetermined. The two types of groups would provide data on the architectural aspects affecting the requirements they are eliciting, and in the end the number of requirements affected by communication protocols would be statistically compared to determine evidence to support or refute the hypothesis. The independent variable is the realization of com- munication protocols, and the dependent variable would be the reported number of affected requirements.
7. Conclusions
The role of an existing software architecture (SA) in require- ments engineering (RE) was recognised as important over a decade ago (Shekaran, 1994a). However, to our knowledge, this issue has not been scientifically explored. This paper describes an exploratory study on this question. This study involved six RE teams eliciting requirements to enhance an existing system, and collect- ing and analyzing data from their in-project decisions that they
made. Collection of data was facilitated by a tool that allowed the teams to not only do their requirements work but also capture study-specific data. This tool was based on a requirements deci- sion meta-model (see Fig. 1) that was designed and validated for use in this study.
2 tems a
•
•
r S t
e c ( c ( c ( a b n n
t t t w s a
A
n
R
B
B
B
B
B
C
C
C
D
454 J.A. Miller et al. / The Journal of Sys
From the findings of the study, we conclude that:
There exist at least four types of architectural effects on RE deci- sions (see Section 4.1.1): as an enabler (30%), as a constraint (25%), as an influence (6%), and as neutral (39%). This means that approximately 60% of the RE decisions were affected (or approximately 40% were not affected) by the SA. These character- istics add significant new knowledge to the literature (Shekaran, 1994b) where the existence of the “constraint” effect was sus- pected but the different types of effects and their extent were not known. Also, different aspects of the SA can have different degrees of effects on RE decisions (see Section 4.1.2). From our study, there were nine different aspects of which “non-functional character- istics (of sub-systems other than the one the analyst is working on for eliciting new requirements)” had the most impact on the affected RE decisions: approximately 29%.
There are several implications of the findings on: planning and isk management; RE and SA technology; architecture evolution; A and RE processes; and Middleware. These are discussed in Sec- ion 5.
Apart from the general need to replicate empirical studies, sev- ral notable suggestions for future empirical work would be to onduct studies based on the following four emergent hypotheses: 1) architects providing “live” feedback to RE agents on system-wide onstraints and enables will reduce amount of requirements-rework, 2) non-functional characteristics of non-local sub-system signifi- antly affect requirements for the local sub-system being worked on, 3) time/effort spent performing evolutionary work in requirements nd architecting processes will decrease if history of interaction effects etween SA and RE is used effectively, and (4) architectural commu- ication protocols used in a current system has a significant effect on ew requirements.
Since ours was only one exploratory study in a particular con- ext, it would be a mistake to generalize these results verbatim o other contexts (Zave, 1997). However, this does not diminish he importance of the findings described in this paper. Instead, e encourage the readers to view this study as an important first
tep for establishing grounded theory for future studies in this rea.
cknowledgement
This work was, in part, supported by Natural Science and Engi- eering Research Council (NSERC) of Canada.
eferences
aldwin, C.Y., Clark, K.B., 2000. Design Rules, vol. 1: The Power of Modularity. The MIT Press.
ass, L., Clements, P., Kazman, R., 2003. Software Architecture in Practice. Addison- Wesley.
oehm, B., 1988. A spiral model of software development and enhancement. IEEE Computer 21 (5), 61–72.
oehm, B., Basili, V., 2001. Software defect reduction top 10 list. IEEE Computer 34 (January (1)), 135–137.
ruin, H.d., Vliet, H.V., 2003. Quality-driven software architecture composition. Jour- nal of Systems and Software 66 (3), 269–284.
arver, J., Shull, F., Basili, V., 2002. Observational studies to accelerate process expe- rience in classroom studies: an evaluation. In: Proc. 2003 Int. Symp. on Emp. Software Engineering (ISESE ‘03), Rome, Italy, December, pp. 72–79.
reswell, J.W., 2003. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications, Thousand Oaks, CA.
ui, X., Sun, Y., Mei, H., 2008. Towards automated solution synthesis and ratio-
nale capture in decision-centric architecture design. In: Proc. Seventh Working IEEE/IFIP Conference on Software Architecture, February, pp. 221–230.
iaz-Pace, A., Hyunwoo, K., Len, B., Philip, B., Felix, B., 2008. Integrating quality attribute reasoning frameworks in the ArchE design assistant. In: Proc. QoSA’08 4th International Conference on the Quality of Software Architecture, University of Karlsruhe (TH), Germany, October 14–17.
nd Software 83 (2010) 2441–2455
Egyed, A., Grunbacher, P., Medvidovic, N., 2001. Refinement and evolution issues in bridging requirements and architecture—the CBSP approach. In: Proc. First International Workshop from Software Requirements to Architectures (STRAW ‘01), Toronto, Canada, June.
El Emam, K., Madhavji, N.H., 1995. Measuring the success of requirements engi- neering processes. In: Proc. 2nd IEEE Int. Symp., RE, York, England, March, pp. 204–211.
Farenhorst, R., Lago, P., Vliet, H.V., 2007. EAGLE: effective tool support for shar- ing architectural knowledge. International Journal of Cooperative Information Systems 16 (3–4), 413–437.
Ferrari, Madhavji, 2008a. Architecting-problems rooted in requirements. Informa- tion and Software Technology 50 (January (1–2)), 53–66.
Ferrari, Madhavji, 2008b. Software architecting without requirements knowledge and experience: what are the repercussions? Journal of Systems and Software 81 (September (9)), 1470–1490.
Garlan, D., 1994. The role of software architecture in requirements engineering. In: Proc. First Int. Conf. on Requirements Engineering, April, p. 240.
Georgakopoulos, D., Hornick, M., Sheth, A., 1995. An overview of workflow manage- ment: from process modeling to workflow automation infrastructure. Journal of Distributed and Parallel Databases 3 (April (2)), 119–153.
Hofmeister, C., Nord, R., Soni, D., 2005. Global analysis: moving from software requirements specification to structural views of the software architecture. IEEE Proceedings Software 152 (August (4)), 187–197.
Host, M., Regnell, B., Wohlin, C., 2000. Using students as subjects—a comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering, 201–214.
IEEE SWEBOK, 2004. Guide to the Software Engineering Body of Knowledge: 2004 Version. IEEE and IEEE Computer Society Project, http://www.swebok.org/.
Jackson, M., 1994. The role of architecture in requirements engineering. In: Proc. First Int. Conf. on Requirements Engineering, April, p. 241.
Johnson, R.B., Christensan, L., 2004. Educational Research: Quantitative, Qualitative and Mixed Approaches, 2nd ed. Allyn & Bacon.
Kazman, R., Klein, M., Clements, P., 2000. ATAM: method for architecture evaluation. Technical Report, Software Engineering Institute, Carnegie Melon University, CMU/SEI-2000-TR-004 ESC-TR-2000-004.
Keuler, T., Muthig, D., Uchida, T., 2008. Efficient quality impact analyses for iterative architecture construction. In: Proc. Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008), pp. 19–28.
Kotonya, G., Sommerville, I., 1998. Requirements Engineering. John Wiley & Sons, Ltd.
Kozaczynski, W., 2002. Requirements, architectures and risks. In: Proc. IEEE Joint International Conference on Requirements Engineering, Essen, Germany, pp. 6–7.
Kruchten, P., 2001. The Rational Unified Process: An Introduction, 2nd ed. Addison- Wesley, Boston.
LaMantia, M.J., Cai, Y., MacCormack, A., Rusnak, J., 2008. Analyzing the evolution of large-scale software systems using design structure matrices and design rule theory: two exploratory cases. In: Proc. Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008), pp. 83–92.
Larman, C., 2003. Agile and Iterative Development: A Manager’s Guide. Addison- Wesley Professional.
Liu, L., Yu, E., 2001. From requirements to architectural design—using goals and scenarios. In: Proc. 2nd Int. Workshop from Soft. Reqts. to Arch. (STRAW ‘01), Toronto, Canada, June.
Miller, J., Ferrari, R., Madhavji, N.H., 2008. Architectural effects on requirements decisions: an exploratory study. In: Proc. 7th Working IEEE/IFIP Confer- ence on Software Architecture (WICSA ‘08), Vancouver, Canada, pp. 231– 240.
Miller, J., Ferrari, R., Madhavji, N.H., 2009. Characteristics of new requirements in the presence or absence of an existing system architecture. In: Proc. 17th IEEE Conference on Requirements Engineering (RE ‘09), Atlanta, USA, August.
Morrison, R., Balasubramaniam, D., Oquendo, F., Warboys, B., Greenwood, R.M., 2007. FIESTA: a generic framework for integrating new functionalities into soft- ware architectures. In: Proc. First European Conference on Software Architecture (ECSA 2007), LNCS 4758, pp. 2–10.
Nuseibeh, B., 2001. Weaving together requirements and architectures. IEEE Com- puters 34 (March (3)), 115–117.
Nuseibeh, B., Easterbrook, S., 2000. Requirements engineering: a roadmap. In: Proc. Conf. on the Future of Software Engineering, ACM Press, pp. 35–46.
Ramesh, B., Jarke, M., 2001. Toward reference models for requirements traceability. IEEE Transactions on Software Engineering 2 (January (1)), 58–93.
Rapanotti, L., Hall, G., Jackson, M., Nuseibeh, B., 2004. Architecture-driven prob- lem decomposition. In: Proc. 12th IEEE International Requirements Engineering Conference (RE 2004), Kyoto, Japan, pp. 80–89.
Runeson, P., 2003. Using students as experiment subjects—an analysis on graduate and freshman student data. In: EASE’03—Proc. 7th Int. Conf. on Empirical Assessment & Evaluation in Software Engineering, April, pp. 95–102.
Schwanke, R., 2005. GEAR: a good enough architectural requirements process. In: Proc. 5th Working IEEE/IFIP Conference on Software Architecture (WICSA 05), Pittsburgh, USA, pp. 57–66.
Shekaran, C., 1994a. Panel overview: the role of software architecture in require- ments engineering. In: Proc. First Int. Conf. on Requirements Engineering, April, p. 239.
tems a
S
S
S
S
S
S
T
T
V
W
W
W
Z
J.A. Miller et al. / The Journal of Sys
hekaran, C., 1994b. The role of software architecture in requirements engi- neering. In: Proc. First Int. Conf. on Requirements Engineering, April, p. 245.
haw, M., 2003. Writing good software engineering research papers: minitutorial. In: Proc. 25th International Conference on Software Engineering (ICSE 2003), Portland, USA, Tutorial Session, pp. 726–736.
hen, Y., Madhavji, N.H., 2006. ESDM—a method for developing evolutionary scenar- ios for analysing the impact of historical changes on architectural elements. In: Proc. 22nd IEEE International Conference on Software Maintenance (ICSM’06), pp. 45–54.
toll, P., Wall, A., Norstrom, C., 2008. Guiding architectural decisions with the influencing factors method. In: Proc. Seventh Working IEEE/IFIP Conference on Software Architecture, February, pp. 179–188.
oftware Requirements to Architectures Workshop (STRAW), 2001. Proc. Interna- tional Conference on Software Engineering (ICSE) Workshop, Toronto, Canada, June.
oftware Requirements to Architectures Workshop (STRAW), 2003. Proc. Interna- tional Conference on Software Engineering (ICSE) Workshop, Portland, USA, May.
ichy, W.F., Lukowicz, Prechelt, L., Ernst, A., 1995. Experimental evaluation in com- puter science: a quantitative study. Journal of Systems and Software (January), 1–18.
rochim, W., 2006. Research Methods Knowledge Base (This is available at http://www.socialresearchmethods.net/kb/design.php. Last accessed January, 2009).
ogt, P., 1993. Dictionary of Statistics and Methodology: A Nontechnical Guide for the Social Sciences. Sage Publications, California, USA.
aignier, G., Le Meur, A.F., Duchien, L., 2007. FIESTA: a generic framework for integrating new functionalities into software architectures. In: Proc. First European Conference on Software Architecture (ECSA 2007), LNCS 4758, pp. 76–91.
ang, Z., Sherdil, K., Madhavji, N., 2005. ACCA: an architecture-centric concern anal- ysis method. In: Proc. IEEE Working Int. Conference on Software Architecture
(WICSA), Pittsburgh, USA, November, pp. 99–108.
ieringa, R.J., Heerkens, J., 2006. The methodological soundness of requirements engineering papers: a conceptual framework and two case studies. Require- ments Engineering Journal 11, 295–307.
ave, P., 1997. Classification of research efforts in requirements engineering. ACM Computing Surveys 29 (4), 315–321.
nd Software 83 (2010) 2441–2455 2455
Nazim H. Madhavji is a Professor in the Department of Computer Science at the University of Western Ontario, Canada. He obtained his Ph.D. from the Univer- sity of Manchester, England, in 1980. His research interests includes: software requirements; software architectures; evolution of software; software quality and measurements; defect tracking and analysis; congruence between software prod- ucts and processes; and empirical studies.
He has led a number of research projects in software engineering, involving corporations such as IBM Canada, DMR Group, CAE Electronics, Transport Canada, and CRIM, and was a Principal Investigator in several multi-university projects. He is the chief architect and editor of the 27-chapter book “Software Evolution and Feedback: Theory and Practice” with Juan F. Ramil and Dewayne Perry, John Wiley, 2006. He is an Editor (with Khaled El Emam) of the book: “Elements of Software Process Assessment and Improvement”, IEEE Computer Society Press, 1999. He is on the Editorial Boards of several scientific journals. He is a consultant to several organisations in the field of software and is a consultant to several universities internationally in the areas of Software Engineering research, pedagogy, and faculty development.
Remo Ferrari is a doctoral candidate at the University of Western Ontario, London, Ontario, Canada. His research interest is in Software Engineering, specifically in the areas of Software Architecture, Requirements and Project Management. In particu- lar, his work has investigated these areas through an empirical viewpoint, examining such issues as the technical effects an architecture has on new requirements, and the impact of human agents background experience on software architecting. His primary research goal is to help in advancing the underlying scientific knowledge and theory with respect to these and related issues. This goal is being pursued in two phases: to first conduct “laboratory” or exploratory studies to generate pre- liminary results and then, based on these findings, to conduct industrial studies. Such empirical work is intended to form underlying grounded theory on which pro- cesses, methods, tools, etc. can be developed. In addition to research, he teaches both Software Engineering and Computer Science courses.
James Miller completed his Master of Science degree at the University of Western
Ontario, London, Ontario, Canada in the area of Software Engineering. His spe- cific research interests are in the areas of Software Architecture and Requirements Engineering, in particular, his investigations in these areas are from an empiri- cal perspective, typically exploratory studies to generate preliminarily results and grounded theory on which further, more confirmatory work can be based. He is currently serving as an officer in the Canadian Air Forces.
- An exploratory study of architectural effects on requirements decisions
- Introduction
- Related work
- RE and SA relationship
- RE–SA technology
- Architecture evolution
- Reflection on research
- The study
- Research questions
- Participants
- The RE project
- The pre-existing requirements document
- The architectural document
- Data collection
- The decision meta-model
- Data-gathering tool
- Data collection
- Threats to validity
- Threats to external validity
- Threats to construct validity
- Threats to conclusion validity
- Results
- How an architecture affects requirements decision-making (Q1)
- Types of architectural effects
- Architectural impact characteristics
- Architectural aspects affecting requirements decisions (Q2)
- Architectural aspects across effect-types
- Architectural aspects across project groups
- Implications
- Planning and risk management
- RE and SA technology
- Architectural evolution
- Tighter SA–RE integration
- RE-to-SA feed-forward process
- Increased middleware
- Analysis
- Future empirical work
- Conclusions
- Acknowledgement
- References
