Introduction to software quality. Quality models and studies for quality: patterns, social, and developers studies. Usefulness of quality models and challenges of multi-language systems.

1. Quality and
Multi-language Systems
Yann-Gaël Guéhéneuc
KAIST
13/10/14
This work is licensed under a Creative
Commons Attribution-NonCommercialShareAlike 3.0 Unported License

2. 2/126

3. Typical software developers?
3/126

4. 4/126

5. Software costs?
5/126

6. 6/126

7. 7/126

8. Cost of bugs
http://www.di.unito.it/~damiani/ariane5rep.html
8/126

9. 9/126

10. 10/126

11. Cost of quality
http://calleam.com/WTPF/?p=4914
11/126

12. Agenda
 Quality
– Quality Models
– Good Practices
– Social Studies
– Developers Studies
 Impact
of Quality Models
 Challenges with Multi-language Systems
12/126

13. qual·i·ty noun ˈkwä-lə-tē
: how good or bad something is
: a characteristic or feature that someone
or something has : something that can
be noticed as a part of a person or thing
: a high level of value or excellence
—Merriam-Webster, 2013
13/126

14. Quality
 Division
of software quality according to
ISO/IEC 9126:2001 and 25000:2005
– Process quality
– Product quality
– Quality in use
14/126

15. Quality
 Division
of software quality according to
ISO/IEC 9126:2001 and 25000:2005
– Process quality
– Product quality
– Quality in use
15/126

16. Quality
 Dimensions
of product quality
– Functional vs. non-functional
• At runtime vs. structural
– Internal vs. external
• Maintainability vs. usability
– Metric-based vs. practice-based
• Objective vs. subjective
16/126

17. Quality
 Dimensions
of product quality
– Functional vs. non-functional
• At runtime vs. structural
– Internal vs. external
• Maintainability vs. usability
– Metric-based vs. practice-based
• Objective vs. subjective
17/126

18. Quality
 Definitions
of internal quality characteristics
– Maintainability
– Flexibility
– Portability
– Re-usability
– Readability
– Testability
– Understandability
18/126

19. Quality
 Definitions
of internal quality characteristics
– Maintainability
– Flexibility
– Portability
– Re-usability
– Readability
– Testability
– Understandability
19/126

20. Maintainability is the ease with which a
software system can be modified
—IEEE Standard Glossary of Software
Engineering Terminology, 2013
20/126

21. Metrics
Quality Models
Models
Good Practices
Definition
Maintainability
Detection
Social Studies
Measures
Occurrences
Characteristics
Factors
Developers Studies
Behaviour
21/126

22. Metrics
Quality Models
Models
Good Practices
Definition
Maintainability
Detection
Social Studies
Measures
Occurrences
Characteristics
Factors
Developers Studies
Behaviour
22/126

23. Quality model are models with the objective
to describe, assess, and–or predict quality
—Deissenboeck et al., WOSQ, 2009
(With minor adaptations)
23/126

24. Quality Models
 Division
of quality models according to
Deissenboeck et al.
– Describe quality characteristics and their
relationships
– Assess the quality characteristics of some
software systems
– Predict the future quality of some software
systems
24/126

25. Quality Models
 Division
of quality models according to
Deissenboeck et al.
– Describe quality characteristics and their
relationships
– Assess the quality characteristics of some
software systems
– Predict the future quality of some software
systems
25/126

26. Quality Models
 Basis
for quality models
– Software measures (aka metrics)
– Relationships among characteristics and metrics
• Theoretical
• Practical
26/126

27. Quality Models
 Metrics
have been well researched
– Chidamber and Kemerer
– Hitz and Montazeri
– Lorenz and Kidd
– McCabe
–…
(Do not miss Briand et al.’s surveys
on cohesion and coupling metrics)
27/126

28. 28/126

29. Who needs models?
29/126

30. 30/126

31. • 130.0 Physics
• 129.0 Mathematics
• 128.5 Computer Science
• 128.0 Economics
• 127.5 Chemical engineering
• 127.0 Material science
• 126.0 Electrical engineering
• 125.5 Mechanical engineering
• 125.0 Philosophy
• 124.0 Chemistry
• 123.0 Earth sciences
• 122.0 Industrial engineering
• 122.0 Civil engineering
• 121.5 Biology
• 120.1 English/literature
• 120.0 Religion/theology
• 119.8 Political science
• 119.7 History
• 118.0 Art history
• 117.7 Anthropology/archeology
• 116.5 Architecture
• 116.0 Business
• 115.0 Sociology
• 114.0 Psychology
• 114.0 Medicine
• 112.0 Communication
• 109.0 Education
• 106.0 Public administration
31/126

32. 32/126

33. 33/126

34. Measures without models
http://hardsci.wordpress.com/2013/09/17/the-hotness-iq-tradeoff-in-academia/
34/126

35. Quality Models
 Different
input metrics, different output
characteristics
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
–…
35/126

36. Quality Models
 Different
input metrics, different output
characteristics
– Bansiya and Davis’ QMOOD
• Design metrics
• Maintainability-related characteristics
– Zimmermann et al.’s models
• Code and historical metrics
• Fault-proneness
–…
36/126

37. Quality Models
 Bansiya
and Davis’ QMOOD
– Characteristics of maintainability
• Effectiveness, extensibility, flexibility, functionality,
reusability, and understandability
– Hierarchical model
• Structural and behavioural design properties of
classes, objects, and their relationships
37/126

38. Quality Models
 Bansiya
and Davis’ QMOOD
– Object-oriented design metrics
• Encapsulation, modularity, coupling, and cohesion…
• 11 metrics in total
– Validation using empirical and expert opinion on
several large commercial systems
• 9 C++ libraries
38/126

39. Quality Models
 Bansiya
and Davis’ QMOOD
39/126

40. Quality Models
 Conclusions
– Sound basis to measure different quality
characteristics
 Limits
– Relation between metrics and quality
characteristics, such as maintainability
– Relation between metrics and what they are
expected to measure
40/126

41. Metrics
Quality Models
Models
Good Practices
Definition
Maintainability
Detection
Social Studies
Measures
Occurrences
Characteristics
Factors
Developers Studies
Behaviour
41/126

42. 42/126

43. Practice, practice
and practice more
43/126

44. 44/126

45. Good Practices
 Maintainers
can follow good practices
– SOLID
– Design patterns
– Design antipatterns
–…
45/126

46. Good Practices
 Maintainers
can follow good practices
– SOLID
– Design patterns
– Design antipatterns
–…
46/126

47. Each pattern describes a problem which
occurs over and over in our environment,
and then describes the core of the solution
to that problem, in such way that you can
use this solution a million times over, without
ever doing it the same way twice.
—Christopher Alexander, 1977
(With minor adaptations)
47/126

48. Good Practices
 Design
patterns
– A general reusable solution to a commonly
occurring problem within a given context in
software design
 Design
antipatterns
– A design pattern that may be commonly used
but is ineffective/counterproductive in practice
48/126

49. Good Practices
49/126

50. Important assumptions
– That patterns can be codified in such a way that
they can be shared between different designers
– That reuse will lead to “better” designs. There is
an obvious question here of what constitutes
“better”, but a key measure is maintainability
—Zhang and Budgen, 2012
(With minor adaptations)
50/126

51. Good Practices
 Pattern
solution = Motif which
connotes an
elegant architecture
51/126

52. Good Practices
 Pattern
solution = Motif which
connotes an
elegant architecture
52/126

53. Good Practices
 Pattern
solution = Motif which
connotes an
elegant architecture
53/126

54. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Good Practices
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
54/126

55. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Good Practices
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
55/126

56. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Good Practices
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
56/126

57. Frame
DrawingEditor
Drawing
Handle
We can identify
in the architecture
of a systems
micro-architectures
similar to
design motifs
to explain the
problem solved
DrawingView
Tool
Good Practices
Panel
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
Figure
AttributeFigure
Component
Client
DecoratorFigure
PolyLineFigure
CompositeFigure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
To compose objects
in a tree-like structure
to describe whole–part
hierarchies
57/126

58. Good Practices
 Conclusions
– Codify experts’ experience
– Help train novice developers
– Help developers’ communicate
– Lead to improved quality
58/126

59. Metrics
Quality Models
Models
Good Practices
Definition
Maintainability
Detection
Social Studies
Measures
Occurrences
Characteristics
Factors
Developers Studies
Behaviour
59/126

60. Social Studies
 Developers’
characteristics
– Gender
– Status
– Expertise
– Training
– Processes
–…
60/126

61. 61/126

62. Agile programming,
anyone?
62/126

63. 63/126

64. Social Studies
 Need
for social studies, typically in the form
of experiments
– Independent variable (few)
– Dependent variables (many)
– Statistical analyses (few)
– Threats to validity (many)
64/126

65. Social Studies
 For
example, impact on identifiers on
program understandability
– Identifier styles [Sharif, Binkley]
– Identifier quality [Lawrie]
– Developers’ gender and identifiers [Sharafi]
–…
65/126

66. Social Studies
 For
example, impact on identifiers on
program understandability
– Identifier styles [Sharif, Binkley]
– Identifier quality [Lawrie]
– Developers’ gender and identifiers [Sharafi]
–…
66/126

67. Social Studies
 Independent
variables
– Gender: male vs. female
– Identifier: camel case vs. underscore
 Dependent
variables
– Accuracy
– Time
– Effort
67/126

68. Social Studies
 Subjects
Subjects’ Demography
(24 Subjects)
Academic background
Gender
Ph.D.
M.Sc.
B.Sc.
Male
Female
11
10
3
15
9
 Conclusions
Accuracy
Time
Effort
68/126

69. Metrics
Quality Models
Models
Good Practices
Definition
Maintainability
Detection
Social Studies
Measures
Occurrences
Characteristics
Factors
Developers Studies
Behaviour
69/126

70. Developers Studies
 Developers’
thought processes
– Cognitive theories
•
•
•
•
Brooks’
Von Mayrhauser’s
Pennington’s
Soloway’s
– Memory theories
•
•
•
•
Kelly’s categories
Minsky’s frames
Piaget’s schema
Schank’s scripts
– Mental models
• Gentner and Stevens’ mental models
70/126

71. Developers Studies

Studying developers’
thought processes
– Yarbus’ eye-movements
and vision studies
– Just and Carpenter’s
eye–mind hypothesis
– Palmer’s vision science
–…
71/126

72. Developers Studies
 Picking
into developers’ thought processes
72/126

73. Developers Studies
 Picking
into developers’ thought processes
73/126

74. Developers Studies
 Picking
into developers’ thought processes
74/126

75. Developers Studies
 Developers’
thought processes
– Reading code
– Reading design models
• Content
• Form
–…
75/126

76. Developers Studies
 Developers’
thought processes
– Reading code
– Reading design models
• Content
• Form
–…
76/126

77. Developers Studies
 Developers’
use of design pattern notations
during program understandability
– Strongly visual [Schauer and Keler]
– Strongly textual [Dong et al.]
– Mixed [Vlissides]
77/126

78. Developers Studies
 Independent
variables
– Design pattern notations
– Tasks: Participation, Composition, Role
 Dependent
variables
– Average fixation duration
– Ratio of fixations
– Ration of fixation times
78/126

79. Developers Studies
 Subjects
– 24 Ph.D. and M.Sc. students
 Conclusions
– Stereotype-enhanced UML diagram [Dong et al.]
more efficient for Composition and Role
– UML collaboration notation and the patternenhanced class diagrams more efficient for
Participation
79/126

80. Feedback Loop
 Use
– Measures
– Occurrences
– Factors
to build “better” quality models
80/126

81. 81/126

82. Modeling for
modeling's sake?
82/126

83. Aristote
384 BC–Mar 7, 322 BC
83/126

84. Aristote
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
84/126

85. Aristote
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
Isaac Newton
Dec 25, 1642–Mar 20, 1727
85/126

86. Aristote
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
Isaac Newton
Dec 25, 1642–Mar 20, 1727
Max Tegmark
May 5, 1967–
86/126

87. Usefulness?
Aristote
384 BC–Mar 7, 322 BC
Galileo Galilei
Feb 15, 1564–Jan 8, 1642
Isaac Newton
Dec 25, 1642–Mar 20, 1727
Max Tegmark
May 5, 1967–
87/126

88. Impact of Quality Models
 DSIV
– SNCF IT department
– 1,000+ employees
– 200+ millions Euros
– Mainframes to assembly to J2EE
– 2003
• Quality team
88/126

89. Impact of Quality Models
 MQL
– Dedicated measurement process
89/126

90. Impact of Quality Models
 MQL
– Web-based reports
90/126

91. Impact of Quality Models
 Independent
variables
– Use of MQL or not
– LOC; team size, maturity, and nature
 Dependent
variables
– Maintainability, evolvability, reusability,
robustness, testability, and architecture quality
– Corrective maintenance effort (in time)
– Proportions of complex/unstructured code and of
commented methods/functions
91/126

92. Impact of Quality Models
 Subjects
– 44 systems
• 22 under MQL (QA=1)
• 22 under ad-hoc processes (QA=0)
92/126

93. Impact of Quality Models
93/126

94. Impact of Quality Models
94/126

95. Impact of Quality Models
95/126

96. Impact of Quality Models
96/126

97. Impact of Quality Models
 Conclusions
– Impact on all dependent variables
– Statistical practical significance
 Limits
– Applicability to today’s software systems
97/126

98. 98/126

99. What’s with
today’s systems?
99/126

100. 100/126

101. 101/126

102. 102/126

103. 103/126

104. 104/126

105. 105/126

106. 106/126

107. 107/126

108. 108/126

109. 109/126

110. Challenges with Multi-language Systems
 Today’s
systems are multi-languages
– Facebook
– Twitter
–…
– Even your “regular” software system is now
multi-language, typically a Web application
110/126

111. Challenges with Multi-language Systems
 New
challenges
– Different computational models
– Complex interfaces (API)
– Wrong assumptions
– Wrong conversions
–…
111/126

112. Challenges with Multi-language Systems
 For
example, control- and data-flows
between Java and “native” C/C++ code
methods in Java are used by Java
classes but (typically) implemented in C/C++
 native
112/126

113. Challenges with Multi-language Systems
 Control-flow
interactions
– Java code calls native code
– Native code calls Java methods
– Native code can “throw” and must catch
exceptions
 Data-flow
interactions
– Java code passes objects (pointers)
– Native code creates objects
–…
113/126

114. Challenges with Multi-language Systems
 Different
computational models
114/126

115. Challenges with Multi-language Systems
 Different
computational models
static void *xmalloc(JNIEnv *env, size_t size) {
void *p = malloc(size);
if (p == NULL)
JNU_ThrowOutOfMemoryError(env, NULL);
return p;
}
#define NEW(type, n) ((type *) xmalloc(env, (n) * sizeof(type)))
static const char *const *splitPath(JNIEnv *env, const char *path) {
...
pathv = NEW(char *, count+1);
pathv[count] = NULL;
...
}
115/126

116. Challenges with Multi-language Systems
 Different
computational models
static void *xmalloc(JNIEnv *env, size_t size) {
void *p = malloc(size);
if (p == NULL)
JNU_ThrowOutOfMemoryError(env, NULL);
return p;
}
#define NEW(type, n) ((type *) xmalloc(env, (n) * sizeof(type)))
static const char *const *splitPath(JNIEnv *env, const char *path) {
...
pathv = NEW(char *, count+1);
pathv[count] = NULL;
...
}
No diversion of control flow!
116/126

117. 117/126

118. Conclusion
118/126

119. 119/126

120. 120/126

121. 121/126

122. 122/126

123. Future directions
123/126

124. 124/126

125. 125/126

126.  Multi-language
system quality characteristics
– Definition and modelling
– Computation
– Characterisation
– Impact
126/126

127. 127/126

128. Good Practices
 Martin
and Feather’s SOLID
– Single responsibility
– Open/closed
– Liskov substitution
– Interface segregation
– Dependency inversion
128/126

129. Good Practices
 What
motifs and what model for these
motifs?
 What
 How
model for the program architecture?
to perform the identification?
129/126

130. Good Practices
 What
motifs and what model for these
motifs?
 What
 How
model for the program architecture?
to perform the identification?
130/126

131. Good Practices
 What
motifs and what model for these
motifs? Design motifs and a
motif meta-model
 What
 How
model for the program architecture?
to perform the identification?
131/126

132. Good Practices
 What
motifs and what model for these
motifs? Design motifs and a
motif meta-model
 What
 How
model for the program architecture?
to perform the identification?
132/126

133. Good Practices
 What
motifs and what model for these
motifs? Design motifs and a
motif meta-model
 What
 How
model for the program architecture?
Design Meta-model
to perform the identification?
133/126

134. Good Practices
 What
motifs and what model for these
motifs? Design motifs and a
motif meta-model
 What
 How
model for the program architecture?
Design Meta-model
to perform the identification?
134/126

135. Good Practices
 What
motifs and what model for these
motifs? Design motifs and a
motif meta-model
 What
 How
model for the program architecture?
Design Meta-model
to perform the identification?
135/126

136. Good Practices
 What
motifs and what model for these
motifs? Design motifs and a
motif meta-model
 What
 How
model for the program architecture?
Design Meta-model
to perform the identification?
136/126

137. Good Practices
 Multi-layer
framework for design motif
identification
 Information
retrieval
– Search space
– Query
– Results
137/126

138. Good Practices
framework
for design motif
identification
Model +
Associations,
aggregations,
and composition
 Multi-layer
Model +
Associations,
aggregations
Model
Code
138/126

139. Good Practices
 Constraint
satisfaction problem solved with
explanation-based constraint programming
(e-CP) to obtain
– No a priori descriptions of variations
– Justification of the identified micro-architectures
– Strong interaction with the developers
139/126

140. Good Practices – Example
 Design
motif (
)
Component
Client
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
140/126

141. Good Practices – Example
 Example
of JHotDraw
– Erich Gamma and Thomas Eggenschwiler
– 2D drawing
– Design patterns
141/126

142. Panel
DrawingEditor
DrawingView
Tool
Drawing
Handle
Good Practices – Example
Frame
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
142/126

143. Panel
DrawingEditor
DrawingView
Tool
Drawing
Handle
Good Practices – Example
Frame
Figure
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
143/126

144. Good Practices – Example
 Micro-architecture
(
)
Figure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
 Maintainability
 Understandability
144/126

145. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
C = {leaf < component, composite <
component, composite component}
D = {DrawingEditor, DrawingView…}
145/126

146. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
C = {leaf < component, composite <
component, composite component}
D = {DrawingEditor, DrawingView…}
146/126

147. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
C = {leaf < component, composite <
component, composite component}
D = {DrawingEditor, DrawingView…}
147/126

148. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
C = {leaf < component, composite <
component, composite component}
D = {DrawingEditor, DrawingView…}
148/126

149. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
C = {leaf < component, composite <
component, composite component}
D = {DrawingEditor, DrawingView…}
149/126

150. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
<
<
C = {leaf < component, composite <
component, composite component}

D = {DrawingEditor, DrawingView…}
150/126

151. Frame
DrawingEditor
Client
Drawing
Handle
Component
DrawingView
Tool
e-CP
Panel
Figure
1..n
operation()
ramification
Leaf
operation()
Composite
add(Component)
remove(Component)
getComponent(int)
operation()
For each components
component.operation()
AbstractFigure
AttributeFigure
DecoratorFigure
PolyLineFigure
CompositeFigure
V = {component, leaf, composite}
<
<
C = {leaf < component, composite <
component, composite component}

D = {DrawingEditor, DrawingView…}
151/126

152. Good Practices


Search space
can
be very large and the
efficiency in time of the
search very low
Use metrics and
topology to reduce
the search space
152/126