In the past decade, numerous experiments of Agile-User Experience Design (also called Agile-UX) have been carried out. Through these experiments it remains unclear who should be in charge of the usability in an Agile-UX project development. After a review of the literature about the involvement of usability expert(s) in Agile-UX, this paper repeats two experiments which explore the necessity to involve usability experts in the team. The first experiment is based on the statement that developers should be able to manage the User-Centred Design (UCD) and conduct the related methods without the intervention of a usability expert, in order to respect agile practices. The second one is based on the statement that integration of a usability expert in project teams ensures better implementation of UCD and better results. Results of both experiments are discussed to validate research hypotheses for future work.

1. Agile-User Experience Design:
With or Without a Usability Expert in the
Team?
Lou Schwartz
Work in progress
ICSEA 2013

2. Agenda
1. Literature review – Usability expert(s) involvement in Agile-UX
2. Research question
3. Experiments
•
With/without usability expert
4. Conclusion
Agile-UX: with or without a usability expert? -- Lou Schwartz
2

3. UCD expert(s) involvement in Agile-UX
Armitage – Chamberlain – Ferreira – McInerney – Nelson – Nodder – Sy
A parallel team of several UCD experts
• Most of the experiments
• Several types of integration
•
•
By spike  not peaceful over the long term
Occasional involvement, only to answer some questions
• UCD experts available throughout the project “on the call”  not possible if he
works on several projects
• No planning
•
Parallel tracks organization of work
• designers work with 1 or 2 iterations ahead of developers
• Several designers needed  amount of work + designer should not be the tester
Agile-UX: with or without a usability expert? -- Lou Schwartz
3

4. UCD expert(s) involvement in Agile-UX
Ferreira – McInerney – Schwartz
1 UCD expert
• Few experiments
• Often he plays also the product owner role
Agile-UX: with or without a usability expert? -- Lou Schwartz
4

5. UCD expert(s) involvement in Agile-UX
Ferreira – McInerney – Nelson – Singh – Sy
UCD expert as product owner
• Product owner responsibilities are near from the UCD
responsibilities and skills
• Often Product owners have not the skills to conduct methods
to address their responsibilities
Product owner responsibilities
UCD expert answers
Define features of the product
Gather data on users, context and
tasks
Responsible of the value of the
product
Context studies taken into account
needs of the organization
Prioritize features
Prioritization is based on users
Can change priorities
Based on new analysis results or on
users’ feedback
Accept or reject work done
Based on evaluation methods like
users tests

6. UCD expert(s) involvement in Agile-UX
Ferreira – McInerney
No UCD experts
 Team members as responsible of the UCD process
Agile-UX: with or without a usability expert? -- Lou Schwartz
6

7. Research question
Why involve a UCD expert in Agile-UX?
Indeed Agile prefers generalists than specialists
In this logic, UCD should be supported by the team & product
owner
… But almost all experiments involve UCD expert(s)
Agile-UX: with or without a usability expert? -- Lou Schwartz
7

8. Statements to test
• S1: without usability expert, if the project team has
sensitivity and some knowledge in UCD, Agile-UX
works.
• S2: with usability expert involved in the project team,
usability of the produced product is better than in
S1.
• S3: the dynamic of the project team is better when a
usability expert is involved.
Agile-UX: with or without a usability expert? -- Lou Schwartz
8

9. Evaluate the suitability of the statements
2 experiments : with & without UCD expert
Context
• Prototyping of mobile application support to construction site
visit done by architects
Agile-UX: with or without a usability expert? -- Lou Schwartz
9

10. Evaluate the suitability of the statements
Use case 1: without
Use case 2: with
The team
The team
• 1 full-time dev UCD skills
• 1 part-time SCRUM master
• 1 part-time Product owner
 business expert
Organization of work
• 6 months
• 22 iterations of 1 week
• SCRUM
•
•
•
•
1 full-time usability expert
1 full-time dev no UCD skills
1 part-time SCRUM master
1 part-time Product owner
 business expert
Organization of work
• 6 months / 2.5 months
• 5 Iterations of 2 weeks
• SCRUM + Sy’s parallel
tracks
Methods used
Methods used
• Brainstorming to build the
product backlog
• Brainstorming to build the
product backlog
• Personas
Agile-UX: with or without a usability expert? -- Lou Schwartz
10

11. Evaluate the suitability of the statements
UCD methods used
Use case 1: without
Personas
Use case 2: with
X
Wire framing
Power point ®
Paper & pen + Balsamiq®
Users’ tests
direct
observation
6 users, 6
scenarios
At every iteration end with 2
users who know the application
and 2 novices
Users’ tests
real situation
1 user during 1
week
-
Satisfaction
questionnaire
SUS
SUS
Focus groups
-
To evaluate the
wireframes
Expert review
-
Bastien & Scapin ergonomics
criteria
Agile-UX: with or without a usability expert? -- Lou Schwartz
11

12. Evaluate the suitability of the statements
Qualitative results
Use case 1: without
Use case 2: with
Feelings of the team
Feelings of the team
• Slow progression
Observed team dynamic
• No real dynamic
• Demotivation
• Quick progression
• Go deeper in the
functionalities proposed
• Improve quality of the
application
Observed team dynamic
• Pair-designing
• Developer increased his
UCD skills
• Good and constant
motivation
Agile-UX: with or without a usability expert? -- Lou Schwartz
12

13. Evaluate the suitability of the statements
Quantitative results
Use case 1: without
Use case 2: with
Users’ satisfaction
Average
Min
Max
75,42 %
62.5 %
90 %
81.25%
75%
92.5%
Number of problems meet
Imp
orta
nce
1
2
3
4
6
8
10
12
15
20
TOTAL
5
2
3
0
1
0
1
1
1
1
15
2
1
1
1
1
1
0
0
0
0
7
Importance =
number of
testers who
met the
problem *
seriousness of
the problem
13

14. Conclusion
Validate that the initial statements are justified
hypotheses for further studies
• S1: without usability expert, if the project team has sensitivity
and some knowledge in HCI, Agile-UX works.
• S2: with usability expert involved in the project team, usability
of the produced product is better than in S1.
• S3: the dynamic of the project team is better when a usability
expert is involved.
Agile-UX: with or without a usability expert? -- Lou Schwartz
14

15. Thank you for your attention
lou.schwartz@tudor.lu