Abstract. [Context/motivation] Quality requirements (QRs) are a concern of both requirement engineering (RE) specialists and software architects (SAs). However, the majority of empirical studies on QRs take the RE analysts’/clients’ perspectives, and only recently very few included the SAs’ perspective. As a result, (i) relatively little is known about SAs’ involvement in QRs engineering and their coping strategies, and (ii) whatever is known mostly comes from small and midsized projects. [Question/problem] The question in this exploratory study is how SAs cope with QRs in the context of large and contract-based software system delivery projects. [Principal ideas/results] We executed an exploratory case study with 20 SAs in the context of interest. The key results indicate the role SAs play in QRs engineering, the type of requirements communication processes SAs are involved in, the ways QRs are discovered, documented, quantified, validated and negotiated. Our most important findings are that in contract-based contexts: (1) the QRs are approached with the same due diligence as the functional requirements and the architecture design demand, (2) the SAs act proactively and embrace responsibilities over the QRs, (3) willingness to pay and affordability seem as important QRs prioritization criteria as cost and benefits do, and (4) QRs engineering is perceived as a social activity and not as much as a tool and method centric activity.[Contribution] The main contributions of the paper are (i) the explication of the QRs process from SAs’ perspective, and (ii) the comparison of our findings with previously published results

1. 1
Software Architects’ Experiences
of Quality Requirements:
What we Know and What we do not Know?
Maya Daneva, Luigi Buglione, Andrea Herrmann

2. 2
Table of Contents
1. Introduction
2. Background and Motivation
3. The Research Design: Qualitative Case
Study
4. The Application of the Method
 Results
 Limitations
1. Comparison to Literature
2. Implications
3. Wrap-Up
Maya Daneva, Wed, April 16, 2009

3. 3
Background and Motivation
 Most empirical research on QRs takes the
perspective of clients, RE researchers,
practitioners.
 Quality requirements (QRs) are a concern of
multiple stakeholders; in particular: software
architects (SAs)
 Relatively little is know about SA’s involvement
 Evidence comes from small and mid-sized
projects; very few studies in large projects

4. 4
The Research Questions
Research Goal: to understand how SAs cope with QRs
in large and contract-based software system
development projects.
1. How do the SAs understand their role?
2. Do SAs and RE staff use different QRs terminology?
3. How do QRs get elicited?
4. How do QRs get documented?
5. How do QRs get prioritized?
6. How do QRs get quantified, if at all?
7. How do QRs get validated?
8. How do QRs get negotiated?
9. What role does the contract play in the way SAs cope
with QRs?

5. 5
The Case Study Research Design
Maya Daneva, Wed, April 22, 2009
Key Steps (R. Yin, 2008):
1. Define interview guide
2. Pilot the interview
3. Collect data by interviewing participants
4. Analyze the data
5. Report on the results

6. 6
Who Did We Interview?
20 Architects from 14 companies in North Europe
 All have 10+ years of experience in large systems
 All work on large contract-based projects (3 dev
locations and 2 client locations)
 Various pricing agreements
 Sectors: large IT vendors, Oil &Gas, Insurance,
Real estate, Video streaming, online systems
(travel, book store, games)

7. 7
Who Did We Analyze the Data?
 Coding practices based on the grounded
theory book of K. Charmaz (2006)
 Iterative procedure

8. 8
Results (1):
How do the SAs understand their role?
 Formal job descriptions and
competence models
 Self-described roles as:
 ‘a bridge’ b/n QRs and underlying
technology
 ‘a translator’ from the user
language into the feature
specification language
 ‘review gate keepers’ regarding
e.g. contract compliance

9. 9
Results (2): Do SAs and RE staff use
different terminology for QRs?
 Gaining communication clarity was a non-
issue
 What helped?
 Domain knowledge
 Experience
 If ISO-certified, then training, quality
manuals, product quality handbooks
made the difference
 Issue: interchangeable use of terms from
two streams of standards (management and
technical how-to)

10. 10
Results (3):
How do QRs get elicited?
 14 SAs use checklists based on:
 ISO standards
 Architecture frameworks
 Internal standards
 Stakeholder engagement
standards, AA1000SES
 4 SAs uses game-based processes
 2 SAs used story-telling techniques

11. 11
Results (4):
How do QRs get documented?
 15 SAs: By using predefined templates
based on :
 ISO standards
 the Quality Function Deployment
framework
 Planguage
 the INVEST grid approach
 Vendor-specific notations (e.g. SAP)
 5 SAs: By using natural language

12. 12
Results (5):
How do QRs get prioritized?
 The business case is the driver
behind trade-offs, e.g. KPIs
 No particular prioritization method
 Prioritization criteria:
 cost,
 benefits,
 client’s willingness to pay
 affordability
 Who decides?
 Steering committees
 SAs

13. 13
Results (6):
How do QRs get quantified?
 Quantification is useful, but should not happen
too early
 How you get them?
 pre-specified in the contract
 Engage a specialist expert (e.g. in
performance)
 Decompose, operationalize and use
estimation technique (IFPUG NFR)
 Issue:
 product and project measures are used
incorrectly
 QRs are confused with design-level req’ts

14. 14
Results (7):
How do QRs get validated?
 By using requirements walkthroughs
 The QFD framework
 Internal architecture standards

15. 15
Results (8):
How do QRs get negotiated?
 The business case is the commonly
used vehicle
 The QFD framework
 EasyWinWin
 The six-thinking-hat method

16. 16
Results (9):
What role does the contract play in the way
SAs cope with QRs?
 3 ways for influencing QRs
processes:
 Cost-conciousness
 Service level agreements
 Pre-defined priorities for QRs
 Contracts are conductive as ways to
maintain control
 3SAs: “contract is was not that
important”
 EasyWinWin

17. 17
Limitations of the Study
The possible validity concerns:
1. External validity
 Similarity with contract-based system delivery
settings
 Application domain, organizational maturity
1. Data collection threats: did SAs answer the
questions truthfully?
2. Data analysis threats: researcher’s bias

18. 18
Comparison to Literature
Input: 5 empirical studies
van Heesch et al, Ameller et al (2) Poort et al (2)
1.QRs are:
 elicited by using checklists,
 documented by means of templates
 prioritized based on willingness to pay and affordability
 quantified by using size estimation standards (IFPUG)
 negotiated by using the business case
1.SAs:
 Serve as ‘a bridge’ and have formal job descriptions,
 Have terminology (established by standards)
1.Contract plays an important role for SAs to act the ways
we observed.

19. 19
Implications
1. To SAs: this study suggests QRs conversations
start with contracts, SLA, and business cases
2. To RE practitioners: your SA could be quite a
valuable resource! She/he can save you time.
3. To RE tool vendors: it seems more important to
figure out how to embed tools into social
processes and broader social interaction
4. To RE researchers: extend the focus on methods,
models and automation by including analysis of
QRs processes as socially constructed ones.

20. 20
Wrap-Up
1. We carried out an interview-based study
2. It revealed what SAs were thinking on QR
3. We compared and contrasted the findings with
published literature
4. Implications fro practice and research are made.

21. 21