1. 21/03/14 pag. 1
Information visualization lecture 5
interaction
Katrien Verbert
Department of Computer Science
Faculty of Science
Vrije Universiteit Brussel
katrien.verbert@vub.ac.be

2. 21/03/14 pag. 2
Where are we?
Fundamentals
Percep'on
Vision
Color
Principles
Techniques
Representa'on
Presenta'on
Interac3on
Applica3ons
Case
studies
Dashboards
Visual
Analy3cs

3. 21/03/14 pag. 3
Interactive data visualizations
Graphic
design
Data
analysis
Sta3c
visualiza3on
Slide
source:
Michael
Porath

4. 21/03/14 pag. 4
Interactive data visualizations
Graphic
design
Data
analysis
Interac'on
Design
Sta3c
visualiza3on
User
interface
design
Exploratory
data
analysis
Interac3ve
visualiza3on
Slide
source:
Michael
Porath

5. 21/03/14 pag. 5
Why interaction?
• When is (static) representation not enough?

6. 21/03/14 pag. 6
Why interaction?
• When is (static) representation not enough?
Scale
• Too
many
data
points
• Too
many
different
dimensions
Storytelling
Explora3on
Learning
Slide
source:
Michael
Porath

7. 21/03/14 pag. 7
User intent: what is the objective?
Select
Reconfigure
Encode
Abstract
/
elaborate
Filter
Connect
Explore
Yi
et
al.
2007

8. 21/03/14 pag. 8
User intent: what is the objective?
Select
Reconfigure
Encode
Abstract
/
elaborate
Filter
Connect
Explore
User
intent

9. 21/03/14 pag. 9
Select / focus
Mark something interesting.

10. 21/03/14 pag. 10
Select/focus

11. 21/03/14 pag. 11
Select/focus
Pick a detail from a larger dataset to keep track of it.

12. 21/03/14 pag. 12
Example: placemark feature in Google Earth

13. 21/03/14 pag. 13
Explore
Show something different.

14. 21/03/14 pag. 14
Explore
example technique panning

15. 21/03/14 pag. 15
Explore
example technique panning

16. 21/03/14 pag. 16
NY Times: Mapping America
• Overcome limitations of display size
• Most common technique: panning
San
Francisco
New
York

17. 21/03/14 pag. 17
h<p://projects.ny'mes.com/census/2010/explorer

18. 21/03/14 pag. 18
Reconfigure
Show a different arrangement.

19. 21/03/14 pag. 19
Reconfigure
h<p://indexity.net/vis/hw/

20. 21/03/14 pag. 20
Encode
Show a different representation.

21. 21/03/14 pag. 21
Change visual/retinal variables
Change
visual
variables
– Colors
– Sizes
– Orienta'on
h<p://bl.ocks.org/mbostock/3943967
– Font
– Shape

22. 21/03/14 pag. 22
Change visual variables
35%
12%
53%
Sales
product
1
product
2
product
3
0
0.1
0.2
0.3
0.4
0.5
0.6
product
1
product
2
product
3
Sales

23. 21/03/14 pag. 23
Abstract/Elaborate
Show more or less detail.

24. 21/03/14 pag. 24
Focus+context
h<p://stats.oecd.org/OECDregionalsta's'cs/
#story=0

25. 21/03/14 pag. 25
Filter
Show something conditionally.

26. 21/03/14 pag. 26
San Francisco crime spotting
Change
the
set
of
data
items
presented
based
on
some
condi'on
h<p://sanfrancisco.crimespoYng.org/

27. 21/03/14 pag. 27
Baby name wizard
h<p://www.babynamewizard.com/voyager

28. 21/03/14 pag. 28
Connect
Show related items.

29. 21/03/14 pag. 29
Brushing and linking

30. 21/03/14 pag. 30
Brushing and linking

31. 21/03/14 pag. 31
OECD Regional Explorer
Mul'ple
views
of
same
data
Selec'ng
or
highligh'ng
in
one
case
generates
highligh'ng
in
another
h<p://stats.oecd.org/OECDregionalsta's'cs/

32. 21/03/14 pag. 32
Interaction vs
Representation

33. 21/03/14 pag. 33
Interaction has an exploratory aspect

34. 21/03/14 pag. 34
The process

35. 21/03/14 pag. 35
Examples
What’s the user intent? Which techniques?

36. 21/03/14 pag. 36
ZIP decode project
h<p://benfry.com/zipdecode/

37. 21/03/14 pag. 37
h<p://www.bloomberg.com/billionaires/2013-­‐03-­‐04/aaa

38. 21/03/14 pag. 38
Where do we like to live?
h<p://www.nieuwsblad.be/ar'cle/detail.aspx?
ar'cleid=DMF20140114_021&__ac'on_ids=10203078490025972&__ac'on_types=o
g.recommends&__source=other_mul'line&ac'on_object_map=[199057810301963]
&ac'on_type_map=[%22og.recommends%22]&ac'on_ref_map

39. 21/03/14 pag. 39
h<p://www.hivegroup.com/solu'ons/demos/usda.html

40. 21/03/14 pag. 40
scenarios

41. 21/03/14 pag. 41
Scenario 1: Minard’s map
• Sta'c
display
• No
physical
interac'on
• Encoded
data
immediately
viewable

42. 21/03/14 pag. 42
Scenario 2: searching for entertainment in London
• discrete information space
• stepped interaction

43. 21/03/14 pag. 43
Scenario 3: estate agent wants to find a house
• Discrete information space
• Continuous interaction
• Responsive system

44. 21/03/14 pag. 44
Scenario 4: coffee table displays advertisements
• Discrete information space
• Moving images
• No physical interaction

45. 21/03/14 pag. 45
Scenario 5: electronic circuit designer effect of
the value of a component
• Continuous interaction
• Continuous relation
• Immediate response
• Preliminary calculation may be needed

46. 21/03/14 pag. 46
Spaces, interaction and
balance of control

47. 21/03/14 pag. 47
Information spaces
Continuous Discrete
Information spaces

48. 21/03/14 pag. 48
Continuous
Stepped
Passive
Composite
Coffee Table
Minard
Entertainment guide
World Wide Web
Function exploration
(Figure 5.5)
Interaction modes
Circuit design (Figure 5.5)
Engineering design (Section 5.4)
Information spaces
Continuous Discrete
Interaction modes

49. 21/03/14 pag. 49
Intent behind interaction
• Exploratory -- e.g. scenario 5 circuit designer
• Seeking -- e.g. scenario 2 entertainment
• Opportunistic -- e.g. ‘see what’s there?’
• Involuntary -- e.g. scenario 4 coffee table

50. 21/03/14 pag. 50
Interaction framework

51. 21/03/14 pag. 51
Form
Intention
Form
Action plan
Execute
Action
Evaluation
Interpretation
Perception
Change in
World
Gulf
of
execution
Gulf
of
evaluation
Goal
Norman’s Action Cycle

52. 21/03/14 pag. 52
Overview
• Scenarios
• Features of interaction
• Interaction framework
– Con3nuous
interac3on
– Stepped
interac'on
– Passive
interac'on
– Composite
interac'on

53. 21/03/14 pag. 53
Ambiguity concerning the
means of placing a limit
Mouse-down only, or
mouse-down and drag?
Continuous interaction

54. 21/03/14 pag. 54
One way of reducing the ambiguity. Mouse-over indicates possible movement
Example to reduce ambiguity

55. 21/03/14 pag. 55
Execution
Display
change
Perception
Interpretation
Evaluation
Stages of the Action Cycle involved in the dynamic
exploration of an effect

56. 21/03/14 pag. 56
A sequence of interactions and the corresponding view changes are interpreted
to form a mental mapping
Display change
Perception
Interpretation
time
about 50 msec
Execution
Display change
Perception
Interpretation
time
about 50 msec
Execution
Sequence of interactions

57. 21/03/14 pag. 57
Circles indicate the qualitative effect, on some overall circuit property, of variation in
the corresponding component
Dynamically triggered pop-out

58. 21/03/14 pag. 58
Second example

59. 21/03/14 pag. 59
Overview
• Scenarios
• Features of interaction
• Interaction framework
– Con'nuous
interac'on
– Stepped
interac3on
– Passive
interac'on
– Composite
interac'on

60. 21/03/14 pag. 60
Stepped interaction in discrete information spaces
We
say
that
the
user
must
navigate
from
one
loca'on
in
discrete
informa'on
space
to
another

61. 21/03/14 pag. 61
Form
Intention
Form
Action plan
Execute
Action
Evaluation
Interpretation
Perception
Change in
World
Gulf
of
execution
Gulf
of
evaluation
Goal
Stepped interaction
• Challenge: support user to form
an action plan
• User must decide which is the
single most beneficial movement
• Questions:
– Where
am
I?
– Where
can
I
go?
–
How
do
I
get
there?
– What
lies
beyond?
– Where
can
I
usefully
go?
+
– Where
have
I
been?

62. 21/03/14 pag. 62
The cloud formed above an island invisible beyond the horizon provides a navigational cue –
suggests what is there and how to get there.
Navigational cue

63. 21/03/14 pag. 63
Navigational cues
We look for cues that will answer:
Where can I go from here?
How do I get there?
Questions refer to
(1) a movement in information space
(2) interaction required to achieve movement
è Defined as sensitivity

64. 21/03/14 pag. 64
Sensitivity
sensitivity: a movement in information space and the
interaction required to achieve it
S=SM, SI
• SM is a single movement in information space
• SI denotes the interaction needed to achieve the movement

65. 21/03/14 pag. 65
The label ‘cafe’ and the flat plate provide navigational cues by showing where the user
can go (the cafe) and how they can get there (push the door)
Illustration

66. 21/03/14 pag. 66
Part of a web page. Each label and surrounding grey area indicate that a mouse
click on the area (SI) will cause movement (SM) to another page concerned with the
selected type of holiday
Illustration II

67. 21/03/14 pag. 67
Interactive visualization and navigation cues
h<p://www.mnh.si.edu/vtp/1-­‐desktop/

68. 21/03/14 pag. 68
Black encoding of houses that fail one attribute limit provides sensitivity information
Example encoding of sensitivity info

69. 21/03/14 pag. 69
In a limit positioning tool colour coding indicates that selection will be unaffected while
the lower limit stays within the white region. When a limit moves into the yellow region
selection will be affected
1 2 3 4 5
Number of bedrooms
Aggregate sensitivity

70. 21/03/14 pag. 70
Limits placed on house attributes by a user leads to the display of houses satisfying
those limits on the map
Price
Number of bedrooms
Journey time
£0k
£50k
£100k
1
2
3
4
5
0
mins
30
mins
Dynamic query interface

71. 21/03/14 pag. 71
A possible modification to the Dynamic Queries interface. Houses violating only one limit
are identified, so that sensitivity is explicit rather than having to be discovered by manual
movement of the limits
Price
lower
limit
upper
limit
Three houses which satisfy all limits with the
sole exception of the upper limit on Price
Sensitivity cues

72. 21/03/14 pag. 72
In the EZChooser outline cars are those that satisfy all requirements except one.
Selection of the range immediately underneath an outline car ensures that the car then
satisfies all requirements
EZChooser

73. 21/03/14 pag. 73
Residue
• Definition of sensitivity has assumed a single movement in information
space – from one page to another
• Normally a succession of pages is visited before a problem is solved.
• The user is continuously asking ‘what lies beyond?’
• Thus a representation of a movement by a cue which additionally
indicates what lies beyond that single movement could enhance
navigation.

74. 21/03/14 pag. 74
Residue
residue: an indication of distant content in the SM encoding
‘distant’ implies content requiring more than one movement

75. 21/03/14 pag. 75
Representation of the top two levels of an hierarchically structured menu-based system
providing information about animals
Mammals
Birds
Fish
Insects
Cats
Bears
Tigers
Whales
Animals
Mammals
Mammals
SM
Insects
SM
Cats
SM
Encoding of SM and
SI for
Mammals
Mammals
Birds
Birds
Fish
Fish
Insects
Insects
Cats
Cats
Bears
Bears
Tigers
Tigers
Whales
Animals
Mammals
Mammals
SM
SM
Cats
SM
Encoding of SM and
SI for
Mammals
Mammals
Birds
Birds
Fish
Fish
Insects
Insects
Cats
Cats
Bears
Bears
Tigers
Tigers
Whales
Animals
Mammals
Mammals
SM
Cats
SM
Encoding of SM and
SI for
Mammals
Mammals
Birds
Birds
Fish
Fish
Insects
Insects
Cats
Cats
Bears
Bears
Tigers
Tigers
Whales
Animals
Mammals
Mammals
SM
Cats
SM
Encoding of SM and
SI for
Mammals
Residue

76. 21/03/14 pag. 76
That part of a 26 menu to be traversed in a successful search for the target word ‘Marlin’
Snowberry
et
al.
(1983)
Science Culture
Biology Technology
Medicine Zoology
Fish Animal
Freshwater Saltwater
Marlin Sailfish
Study on effect of hierarchy structure

77. 21/03/14 pag. 77
1
2
3
4
5
6
10
20
30
40
Percent
total
error
Menu level
Errors made at different levels of a narrow and deep six-
level menu in the search for a target at the lowest level

78. 21/03/14 pag. 78
50
60
70
80
90
100
Percentcorrectsearch
Number of options displayed at each level
2
4
8
64
Percentage correct search as a function of menu structure

79. 21/03/14 pag. 79
Example of the provision of an ‘Upcoming’ help field, where samples from the next lower
level help to enhance confidence in the interpretation of the menu options (Snowberry et
al. 1985)
Help fields

80. 21/03/14 pag. 80
Scent
So far: design cues that encode sensitivity and distant content
User must assess benefit of each available movement, not only
asking:
Where can I go from here?
but crucially,
Where can I most beneficially go from here?
è defined as scent (Pirolli and Card, 1999)

81. 21/03/14 pag. 81
Scent
scent: the perceived benefit associated with a movement in the
information space, evaluated following the interpretation of one
or more cues.

82. 21/03/14 pag. 82
remote content
Residue
(= cue encoding
remote content)
sensitivity cues
SM and SI
encoding
Interpreted
sensitivity cues
and residues
scent
human evaluation
of the benefit of
available SMs
human interpretation
Relation between sensitivity, residue and scent

83. 21/03/14 pag. 83
Personnel
Admin
Research
Sales
Marketing
Encoding to support the evaluation of scent

84. 21/03/14 pag. 84
Distance of a black house to a limit may influence the choice of limit adjustment
Encoding to support the evaluation of scent

85. 21/03/14 pag. 85
Where am I?
• Questions so far:
Where
can
I
go
from
here?
Do
do
I
get
there?
What
lies
beyond?
Where
can
I
usefully
go
from
here?
• Next:
Where
am
I?
and
Where
have
I
been,
because
I
may
want
to
go
back?

86. 21/03/14 pag. 86
Breadcrumbs
• Represent history
• From the story Hans and Gretel
• Two types:
– Path
breadcrumbs
– Loca'on
breadcrumbs

87. 21/03/14 pag. 87
A representation of history leading to the current location
User’s
path
Current
location
Path breadcrumbs

88. 21/03/14 pag. 88
http://news.bbc.co.uk
http://news.bbc.co.uk/1/hi/education/default.stm
http://news.bbc.co.uk/1/hi/england/south_counties/4932646.stm
http://news.bbc.co.uk/1/hi/england/south_counties/4892000.stm
Back
An ordered list of recently visited URLs

89. 21/03/14 pag. 89
Back
An ordered collection of miniatures of recently visited web
pages may provide useful navigational cues

90. 21/03/14 pag. 90
Userʼs
path
Current
location
Outlinks
Available paths from the current location in
discrete information space

91. 21/03/14 pag. 91
Location breadcrumbs (red) provide an awareness of the structure of a site within
which the current location resides
Location
structure
Userʼs
path
Location breadcrumbs

92. 21/03/14 pag. 92
An example of path breadcrumbs within a website

93. 21/03/14 pag. 93
Two menu systems investigated by Field and Apperley (1990)

94. 21/03/14 pag. 94
Critiquing websites
h<p://deredac'e.be/cm/vrtnieuws

95. 21/03/14 pag. 95

96. 21/03/14 pag. 96

97. 21/03/14 pag. 97
Overview
• Scenarios
• Features of interaction
• Interaction framework
– Con'nuous
interac'on
– Stepped
interac'on
– Passive
interac3on
– Composite
interac'on

98. 21/03/14 pag. 98
Two important aspects
of passive interaction
1. During typical use of a visualization tool, most of the user’s
time is spent on passive interaction – often involving eye
movement
2. Passive interaction does not imply a static representation

99. 21/03/14 pag. 99
Static display
Would you go to India just to see if you wanted to be there?

100. 21/03/14 pag. 100
Static representation of outline cars that satisfy all requirements except one: engenders
a ‘see and go’ approach rather that a ‘go and see’ approach
Static display

101. 21/03/14 pag. 101
Visual browsing undertaken by a person with an interest in books on cognitive psychology, on
approaching a book display. The first regional focus (red) explores the entire collection to establish
a new focus (green) associated with psychology. As a result of the exploration of the green region a
new region of focus (blue) is established concerned with cognitive psychology.
First regional focus Second regional focus
Third regional focus
Visual interaction

102. 21/03/14 pag. 102
A continuous sequence of
representations of the US dollar
– euro exchange rate
Moving displays

103. 21/03/14 pag. 103
parameter
values
Discrepancy between
desired and achieved
quality
Sketch of the ‘cockpit’ of a computer-aided circuit
design system

104. 21/03/14 pag. 104
Norman’s Action Cycle for involuntary browsing
Form
Intention
Form
Action plan
Execute
Action
Evaluation
Interpretation
Perception
Change in
World
Goal
Gulf
of
evaluation
Form
Intention
Form
Action plan
Execute
Action
Form
Intention
Form
Action plan
Execute
Action
Evaluation
Interpretation
Perception
Change in
World
Goal
Gulf
of
evaluation
Change
in World
Change
in World

105. 21/03/14 pag. 105
Overview
• Scenarios
• Features of interaction
• Interaction framework
– Con'nuous
interac'on
– Stepped
interac'on
– Passive
interac'on
– Composite
interac3on

106. 21/03/14 pag. 106
A number of randomly generated designs in parameter space (a) are simulated and the
corresponding properties displayed in performance space (b). If acceptable performance
is identified in performance space (b) the corresponding designs can be traced back to
parameter space (a)
Influences

107. 21/03/14 pag. 107
Generalized selection via interactive query
relaxation (Heer et al. 2008)
h<p://vis.berkeley.edu/papers/generalized_selec'on/

108. 21/03/14 pag. 108
Structure described by 4 parameters and 4
performances S1, S2, S3 and S4

109. 21/03/14 pag. 109
Limits placed on the four stresses S1 to S4 have been brushed into the parameter
histograms, with red designs indicating those which satisfy all limits on S1, S2, S3 and S4

110. 21/03/14 pag. 110
Prosection matrix

111. 21/03/14 pag. 111
With information visualization, Norman’s ‘change in world’ consists of an old view of data
being replaced by a new view
PerceptionExecute
Action
Change in World
Old
view
New
view
Interaction dynamics

112. 21/03/14 pag. 112
Cone tree

113. 21/03/14 pag. 113
Replacement of one representation (a) by another (b) might best be achieved by
animation through the representations of (c)
A
B
C
D
E
F
Country
(a) (b)
(c)
Interaction dynamics

114. 21/03/14 pag. 114
Interaction
concepts and
techniques
Sensitivity
Norman’s Cycle
Affordances
Navigation
Residue
Scent
Visual dynamics
Interaction modes
Recap

115. 21/03/14 pag. 115
Readings
• Chapter 5

116. 21/03/14 pag. 116
References
• Field, G. E., & Apperley, M. D. (1990). Context and selective retreat in
hierarchical menu structures. Behaviour & Information Technology, 9(2),
133-146.
• Heer, J., Agrawala, M., & Willett, W. (2008, April). Generalized selection
via interactive query relaxation. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems (pp. 959-968).
ACM.
• Pirolli, P., & Card, S. (1999). Information foraging. Psychological review,
106(4), 643.
• Snowberry, K., Parkinson, S. R., & Sisson, N. (1983). Computer display
menus. Ergonomics, 26(7), 699-712.
• Snowberry, K., Parkinson, S., & Sisson, N. (1985). Effects of help fields
on navigating through hierarchical menu structures. International Journal
of Man-Machine Studies, 22(4), 479-491.
• Yi, J. S., ah Kang, Y., Stasko, J. T., & Jacko, J. A. (2007). Toward a
deeper understanding of the role of interaction in information
visualization. Visualization and Computer Graphics, IEEE Transactions
on, 13(6), 1224-1231.