In this first lecture, we discuss software quality, introduce the quality characteristic of maintainability, and argue that maintainability can be studied from four different points of view: (1) quality models, (2) good practices, (3) social studies, and (4) developers' studies. We discuss major works and results for these four points of view and show the last three can be used in the first one to build better quality models. We show that quality models are mandatory to make sense of any quality evaluation.
1. Some Theory and Practice on
Patterns – Introduction
Yann-Gaël Guéhéneuc
NII, Tokyo, Japan
12/02/14
This work is licensed under a Creative
Commons Attribution-NonCommercialShareAlike 3.0 Unported License
12. Agenda
Quality
Maintainability
– Quality Models
– Good Practices
– Social Studies
– Developers Studies
Impact
of Quality Models
Challenges with Multi-language Systems
12/155
13. “qual·i·ty noun ˈkwä-lə-tē
how good or bad something is
a characteristic or feature that someone
or something has : something that can
be noticed as a part of a person or thing
a high level of value or excellence”
—Merriam-Webster, 2013
13/155
14. Quality
Division
of software quality according to
ISO/IEC 9126:2001, 25000:2005…
– Process quality
– Product quality
– Quality in use
http://www.sqa.net/iso9126.html
14/155
15. Quality
Division
of software quality according to
ISO/IEC 9126:2001, 25000:2005…
– Process quality
– Product quality
– Quality in use
15/155
16. Quality
Dimensions
of product quality
– Functional vs. non-functional
• At runtime vs. structural
– Internal vs. external
• Maintainability vs. usability
– Metric-based vs. practice-based
• Objective vs. subjective
16/155
17. Quality
Dimensions
of product quality
– Functional vs. non-functional
• At runtime vs. structural
– Internal vs. external
• Maintainability vs. usability
– Metric-based vs. practice-based
• Objective vs. subjective
17/155
20. “Maintainability is the ease with which a
software system can be modified”
—IEEE Standard Glossary of Software
Engineering Terminology, 2013
20/155
23. “Quality model are models with the objective
to describe, assess, and–or predict quality”
—Deissenboeck et al., WOSQ, 2009
(With minor adaptations)
Florian Deissenboeck, Elmar Juergens, Klaus Lochmann, and Stefan Wagner ;
Software Quality Models: Purposes, Usage Scenarios and Requirements ;
International Workshop on Software Quality,
24th International Symposium on Computer and Information Sciences, IEEE, 2009
23/155
24. Quality Models
Division
of quality models according to
Deissenboeck et al.
– Describe quality characteristics and their
relationships
– Assess the quality characteristics of some
software systems
– Predict the future quality of some software
systems
24/155
25. Quality Models
Division
of quality models according to
Deissenboeck et al.
– Describe quality characteristics and their
relationships
– Assess the quality characteristics of some
software systems
– Predict the future quality of some software
systems
25/155
26. Quality Models
Basis
for quality models
– Software measures (aka metrics)
– Relationships among characteristics and metrics
• Theoretical
• Practical
26/155
27. Quality Models
Metrics
have been well researched
– Chidamber and Kemerer
– Hitz and Montazeri
– Lorenz and Kidd
– McCabe
–…
27/155
28. Quality Models
Typical
kinds of metrics
– Size
– Coupling
– Cohesion
• Briand et al.’s surveys on cohesion and coupling
– Complexity
Lionel C. Briand, John W. Daly, and Jürgen Wüst ;
A Unified Framework for Cohesion Measurement in Object-OrientedSystems ;
Journal of Empirical Software Engineering, 3(1), Springer, 1998
Lionel C. Briand, John W. Daly, and Jürgen Wüst ;
A Unified Framework for Coupling Measurement in Object-Oriented Systems ;
Transactions on Software Engineering, 25(1), IEEE Press, 1999
28/155
29. Quality Models
Typical
metrics
– LOC, fan-in and fan-out
– LCOM5
– CBO
– McCabe’s complexity
Typically
computed automatically on source
code or other intermediate representations
29/155
38. Quality Models
Different
input metrics, output characteristics
– Menzies et al.’s models
• Code metrics
• Defectiveness
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
38/155
39. Quality Models
Different
input metrics, output characteristics
– Menzies et al.’s models
• Code metrics
• Defectiveness
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
39/155
40. Quality Models
Menzies
et al.’s models
– Characteristics of defectiveness
• Was the class/module involved in one fault or more?
– Three kinds of models
• OneR
• J48
• Naïve Bayesian miner
Tim Menzies, Member, IEEE, Jeremy Greenwald, and Art Frank ;
Data Mining Static Code Attributes to Learn Defect Predictors ;
Transactions on Software Engineering, 33(1), IEEE, 2007
40/155
41. Quality Models
Menzies
et al.’s models
– Code metrics
• McCabe’s cyclomatic, design, essential complexities
• LOC (total, blanks, comments…)
• Halstead’s numbers of operands and operators (and
variations and combinations)
– Collection and validation using NASA MDP
• 8 systems in C and Java
(No source code)
http://nasa-softwaredefectdatasets.wikispaces.com/
41/155
43. Quality Models
Different
input metrics, output characteristics
– Menzies et al.’s models
• Code metrics
• Defectiveness
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
43/155
44. Quality Models
Different
input metrics, output characteristics
– Menzies et al.’s models
• Code metrics
• Defectiveness
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
44/155
45. Quality Models
Different
input metrics, output characteristics
– Menzies et al.’s models
• Code metrics
• Defectiveness
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
45/155
46. Quality Models
Bansiya
and Davis’ QMOOD
– Characteristics of maintainability
• Effectiveness, extensibility, flexibility, functionality,
reusability, and understandability
– Hierarchical model
• Structural and behavioural design properties of
classes, objects, and their relationships
Jagdish Bansiya , Carl G. Davis ;
A Hierarchical Model for Object-oriented Design Quality Assessment ;
Transactions on Software Engineering, 28(1), IEEE, 2002
46/155
47. Quality Models
Bansiya
and Davis’ QMOOD
– Object-oriented design metrics
• Encapsulation, modularity, coupling, and cohesion…
• 11 metrics in total
– Validation using empirical and expert opinion on
several large commercial systems
• 9 C++ libraries
(Source code)
47/155
49. Quality Models
Conclusions
– Sound basis to measure different quality
characteristics
Limits
– Relation between metrics and quality
characteristics, such as maintainability
– Relation between metrics and what they are
expected to measure
49/155
50. Quality Models
Limits
– Relation between metrics and quality
characteristics, such as maintainability
– Relation between metrics and what they are
expected to measure
Overcome
by using more, diverse,
unrelated information
50/155
58. “Each pattern describes a problem
which occurs over and over in our
environment, and then describes
the core of the solution to that
problem, in such way that you can
use this solution a million times
over, without ever doing it the
same way twice.”
—Christopher Alexander, 1977
(With minor adaptations)
58/155
59. Good Practices
Design
patterns
– A general reusable solution to a commonly
occurring problem within a given context in
software design
Design
antipatterns
– A design pattern that may be commonly used
but is ineffective/counterproductive in practice
59/155
61. Design Patterns
“Important assumptions
– That patterns can be codified in such a way that
they can be shared between different designers
– That reuse will lead to “better” designs. There is
an obvious question here of what constitutes
“better”, but a key measure is maintainability”
—Zhang and Budgen, 2012
(With minor adaptations)
61/155
65. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Design Patterns
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
65/155
66. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Design Patterns
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
66/155
67. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Design Patterns
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
67/155
68. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Design Patterns
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
68/155
69. Design Patterns
Conclusions
– Codify experts’ experience
– Help train novice developers
– Help developers’ communicate
– Lead to improved quality
69/155
70. Design Anti-patterns
Important
assumptions
– Poor design choices that are conjectured to
make object-oriented systems harder to
maintain
– Yet, maybe the best and possibly only way to
implement some requirements and–or some
functionalities
70/155
71. Design Anti-patterns
Problem
– Problem recurring in object-oriented design
Poor
solution
– Initially may look like a good idea
Alternative
solution
– Repeatable (design pattern)
– Effective
71/155
73. Design Anti-patterns
Conclusions
– Codify experts’ “bad” experience
– Help train novice developers
– Help developers’ communicate
– Lead to improved quality
73/155
74. Good Practices
Limits
– So many patterns
– So many
•
•
•
•
Programming languages
Development contexts
Application domains
Expertises
How
to use their occurrences in a model?
74/155
80. Social Studies
Need
for social studies, typically in the form
of experiments
– Independent variable (few)
– Dependent variables (many)
– Statistical analyses (few)
– Threats to validity (many)
80/155
81. Social Studies
For
example, impact on identifiers on
program understandability
– Identifier styles [Sharif, Binkley]
– Identifier quality [Lawrie]
– Developers’ gender and identifiers [Sharafi]
–…
81/155
82. Social Studies
For
example, impact on identifiers on
program understandability
– Identifier styles [Sharif, Binkley]
– Identifier quality [Lawrie]
– Developers’ gender and identifiers [Sharafi]
–…
Zohreh Sharafi, Zéphyrin Soh, Yann-Gaël Guéhéneuc, and Giuliano Antoniol ;
Women & Men: Different but Equal – A Study on the Impact of Identifiers on Source Code Underestanding ;
Proceeding of 20th International Conference on Program Comprehension, IEEE, 2012
82/155
83. Social Studies
Independent
variables
– Gender: male vs. female
– Identifier: camel case vs. underscore
Dependent
variables
– Accuracy
– Time
– Effort
83/155
84. Social Studies
Subjects
Subjects’ Demography
(24 Subjects)
Academic background
Gender
Ph.D.
M.Sc.
B.Sc.
Male
Female
11
10
3
15
9
Conclusions
Accuracy
Time
Effort
84/155
93. Developers Studies
Developers’
thought processes
– Reading code
– Reading design models [Cepeda]
• Content
• Form
–…
Gerardo Cepeda Porras and Yann-Gaël Guéhéneuc ;
An empirical study on the efficiency of different design pattern representations in UML class diagrams ;
Journal of Empirical Software Engineering, 15(5), Springer, 2010
93/155
94. Developers Studies
Developers’
use of design pattern notations
during program understandability
– Strongly visual [Schauer and Keler]
– Strongly textual [Dong et al.]
– Mixed [Vlissides]
94/155
95. Developers Studies
Independent
variables
– Design pattern notations
– Tasks: Participation, Composition, Role
Dependent
variables
– Average fixation duration
– Ratio of fixations
– Ration of fixation times
95/155
96. Developers Studies
Subjects
– 24 Ph.D. and M.Sc. students
Conclusions
– Stereotype-enhanced UML diagram [Dong et al.]
more efficient for Composition and Role
– UML collaboration notation and the patternenhanced class diagrams more efficient for
Participation
96/155
98. Impact of Quality Models
Usefulness
of the feedback?
Usefulness of the models?
98/155
99. Feedback?
Trend
to use more, diverse, unrelated
information as input of quality models
– Code source metrics
– Historical metrics
– Design metrics
99/155
100. Feedback?
Trend
to use more, diverse, unrelated
information as input of quality models
– Code source metrics
– Historical metrics
– Design metrics
– Good practices
– Social studies
– Developers’ studies
100/155
105. Aristotle
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
Isaac Newton
Dec 25, 1642–Mar 20, 1727
105/155
106. Aristotle
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
Isaac Newton
Dec 25, 1642–Mar 20, 1727
Max Tegmark
May 5, 1967–
106/155
107. Usefulness?
Aristotle
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
Isaac Newton
Dec 25, 1642–Mar 20, 1727
Max Tegmark
May 5, 1967–
107/155
108. Usefulness?
DSIV
– SNCF IT department
– 1,000+ employees
– 200+ millions Euros
– Mainframes to assembly to J2EE
– 2003
• Quality team
Houari Sahraoui, Lionel C. Briand, Yann-Gaël Guéhéneuc , and Olivier Beaurepaire ;
Investigating the impact of a measurement program on software quality ;
Journal of Information and Software Technology, 52(9), Elsevier, 2010
108/155
111. Usefulness?
Independent
variables
– Use of MQL or not
– LOC; team size, maturity, and nature
Dependent
variables
– Maintainability, evolvability, reusability,
robustness, testability, and architecture quality
– Corrective maintenance effort (in time)
– Proportions of complex/unstructured code and of
commented methods/functions
111/155
130. Multi-language Systems
Today’s
systems are multi-languages
– Facebook
– Twitter
–…
– Even your “regular” software system is now
multi-language, typically a Web application
130/155
132. Multi-language Systems
For
example, control- and data-flows
between Java and “native” C/C++ code
methods in Java are used by Java
classes but (typically) implemented in C/C++
native
Gang Tan and Jason Croft ;
An empirical security study of the native code in the JDK ;
Proceedings of the 17th Security Symposium, USENIX Association, 2008
132/155
142. Challenges
Not
only for quality assurance!
(Just for examples…)
Build systems
descriptions
142/155
143. Challenges
Not
only for quality assurance!
(Just for examples…)
Build systems
descriptions
Legal issues due
to interrelations
143/155
144. Challenges
Not
only for quality assurance!
(Just for examples…)
Build systems
descriptions
Legal issues due
to interrelations
Clone definition
and detection
144/155
145. Challenges
Not
only for quality assurance!
(Just for examples…)
Build systems
descriptions
Legal issues due
to interrelations
Clone definition
and detection
Impact on studies
of large systems
145/155