Workshop at the JTEL SS in Cyprus - together with Sten Govaerts & Erik Duval

1. PIMPYOUR LEARNING ANALYTICS
WITH PROPER INFORMATION
VISUALISATION
Joris Klerkx
joris.klerkx@cs.kuleuven.be
JTEL Summerschool 2013, Limassol
Sten Govaerts Erik Duval
erik.duval@cs.kuleuven.besten.govaerts@epfl.ch
Wednesday 5 June 13

2. Actionable dashboards
Wednesday 5 June 13

3. Some design basicsgraph
visualization
On the menu.
Create your own learning analytics dashboard
Wednesday 5 June 13

4. Find out what a data set is about
Information Visualisation is the use of interactive
visual representations to amplify cognition [Card. et. al]
What are the stories behind the data?
Communicating data
Facilitate human interaction for exploration and understanding
Empower people to make informed decisions and
change their learning behaviour
Discover patterns & errors in the data
Wednesday 5 June 13

5. Anscombe’s quartet
•uX = 9.0
•uY = 7.5
• sigma X = 3.317
• sigma Y = 2.03
• Y = 3 + 0.5X
• R2 = 0.67
Find the patterns...
Wednesday 5 June 13

6. Anscombe’s quartet
•uX = 9.0
•uY = 7.5
• sigma X = 3.317
• sigma Y = 2.03
• Y = 3 + 0.5X
• R2 = 0.67
Wednesday 5 June 13

7. http://www.datavis.ca/milestones/
Not new.
Wednesday 5 June 13

8. Some facts to keep in mind.
Wednesday 5 June 13

9. Our brains makes us extremely good at recognizing visual patterns
Humans have advanced perceptual abilities.
Wednesday 5 June 13

10. Our brains makes us extremely good at recognizing visual patterns
Humans have advanced perceptual abilities.
Wednesday 5 June 13

11. Humans have little short term memory
Our brains remember relatively little of what we perceive
Make it interactive, provide visual help
Wednesday 5 June 13

12. Real data is ugly and needs to be cleaned.
Pre-process your data with existing tools, eg. google refine
http://www.netmagazine.com/features/seven-dirty-secrets-data-visualisation
https://code.google.com/p/google-refine/
Wednesday 5 June 13

13. Which of these line graphs is easier to read?
Forget about 3D graphs, we see in 2,05D
Occlusion.
Complex to interact with.
Doesn’t add anything to the story.
"
"
"
"
"
"
"
"
"
"
"
5/27/1
5/28/1
5/29/1
5/30/1
5/31/1
6/1/13"
6/2/13"
6/3/13"
6/4/13"
6/5/13"
6/6/13"
6/7/13"
Student"1"
Student"2""
Student"3"
Student"4"
Student'1'
Student'3'
0'
5'
10'
15'
20'
25'
30'
35'
40'
45'
50'
5/27/13'
5/28/13'
5/29/13'
5/30/13'
5/31/13'
6/1/13'
6/2/13'
6/3/13'
6/4/13'
6/5/13'
6/6/13'
6/7/13'
Student'1'
Student'2''
Student'3'
Student'4'
Cumulative hours spent in a course Cumulative hours spent in a course
Wednesday 5 June 13

14. Size & angle are not pre-attentive: difficult to compare
Limited short term (visual) memory
Who wrote more blogposts? Student 1 or student 4?
blogposts(
Student'1'
Student'2'
Student'3'
Student'4'
8"
10"
10"
10"
blogposts(
Student"1"
Student"2"
Student"3"
Student"4"
blogposts(
Student'1'
Student'2'
Student'3'
Student'4'
Forget about 3D graphs
“Save the pies for dessert” S. Few
Wednesday 5 June 13

15. Use pre-attentive characteristics
Ability of low-level human visual system
to rapidly identify certain basic visual properties
http://www.csc.ncsu.edu/faculty/healey/PP/
e.g. find yourself as student
Be careful with combinations (serial search)
Wednesday 5 June 13

16. USE COMMON SENSE
0"
5"
10"
15"
20"
25"
30"
tweets" blogposts" comments"on"blogs" Reports"submi7ed"
Amount'
Student activity
0"
5"
10"
15"
20"
25"
30"
blogposts" tweets" comments"on"blogs" Reports"submi7ed"
Amount'
0"
5"
10"
15"
20"
25"
30"
blogposts" tweets" comments"on"blogs" Reports"submi7ed"
Amount'
Wednesday 5 June 13

17. Provides some
impressions about
student activity
0"
10"
20"
30"
40"
50"
60"
Student"1" Student"2" Student"3" Student"4"
Reports"submi7ed"
comments"on"blogs"
tweets"
blogposts"
0%#
10%#
20%#
30%#
40%#
50%#
60%#
70%#
80%#
90%#
100%#
Student#1# Student#2# Student#3# Student#4#
Reports#submi;ed#
comments#on#blogs#
tweets#
blogposts#
What/how are you comparing?
What story do you get from it?
Wednesday 5 June 13

18. 0" 5" 10" 15" 20" 25"
Student"1"
Student"2"
Student"3"
Student"4"
comments(on(blogs(
Provides impressions
about student activity
Provides actual values
0" 5" 10" 15" 20"
Student"1"
Student"2"
Student"3"
Student"4"
blogposts(
0" 5" 10" 15" 20" 25" 30" 35"
1"
2"
3"
4"
tweets%
+
Coordinated graphs
Wednesday 5 June 13

19. Step 1: Formulate initial questions
“where” “when’’ “how much” “how many” “How often” (“why”)
Who are your intended users? (teachers? students? researchers?)
Wednesday 5 June 13

20. Step 2: Understand the dataset
Define the characteristics of the data
Time? hierarchical? 1D? 2D? nD? network data?
Quantitative, Ordinal, Categorical?
S. Stevens “On the theory of scales and measurements” (1946)
Wednesday 5 June 13

21. Encode data points into visual form
Step 3: Apply a visual mapping
Simplicity is the ultimate sophistication.
Leonardo da Vinci
Each mark (point, line, are, ...) represents a
data element
Think about relationships between elements
Wednesday 5 June 13

22. Size
most commonly used (?)
Wednesday 5 June 13

23. used for identifying patterns & anomalies in big datasets
Colors
Use maximum +/- 5 colors (for categories,.. )
Use colorbrewer2.org to select appropriate color scheme
Every square = student
colors: progress in course
Wednesday 5 June 13

24. ¡ Law
of
Proximity
The closer objects are to each other,
the more likely they are to be
perceived as a group (Ehrenstein,
2004)
Gestalt Principles
http://www.slideshare.net/chelsc/gestalt-laws-and-design-presentation
Eg. student interests
Wednesday 5 June 13

25. ¡ Law
of
Similarity
Objects that are similar, with like
components or attributes are more
likely to be organised together
(Schamber, 1986).
Objects are viewed in vertical rows because
of their similar attributes.
¡ Law
of
Common
Fate
Objects with a common movement, that move
in the same direction, at the same pace , at the
same time are organised as a group
(Ehrenstein, 2004).
Gestalt Principles
http://www.slideshare.net/chelsc/gestalt-laws-and-design-presentation
Wednesday 5 June 13

26. ¡ Law
of
Isomorphism
Is similarity that can be behavioural or
perceptual, and can be a response based
on the viewers previous experiences
(Luchins & Luchins, 1999; Chang, 2002).
This law is the basis for symbolism
(Schamber, 1986).
There are more!
Gestalt Principles
http://www.slideshare.net/chelsc/gestalt-laws-and-design-presentation
Wednesday 5 June 13

27. SOME HELP
Wednesday 5 June 13

28. SOME HELP
Wednesday 5 June 13

29. http://jlsantoso.blogspot.com/2013/05/reveal-it-applied-in-educational-context.html
Example 1
Wednesday 5 June 13

30. Example 2
Wednesday 5 June 13

31. Example 3 (not Learning analytics but just nice...)
Wednesday 5 June 13

32. Some design basicsgraph
visualization
Create your own learning analytics dashboard
TODAYs workshop
Wednesday 5 June 13

33. Form teams of 4-5 persons
GOAL: Sketch your own learning analytics
dashboard on paper
Wednesday 5 June 13

34. we created a model...
Wednesday 5 June 13

35. Wednesday 5 June 13

36. you can extend/change it!
Wednesday 5 June 13

37. It’s time to work!
Wednesday 5 June 13

38. Step 3: Apply a visual mapping to your dataset
Sketch on paper
(Step 4: Think about interaction of visualisation app)
e.g. what kind of filtering mechanisms?
Step 2: Understand the dataset
Data characteristics (‘actionable’ ones)
Step 1: Formulate initial questions
“where” “when’’ “how much” “how many” “How often” (“why”)
Who are your intended users? (teachers? students? researchers?)
Step 5: How to evaluate visualisations?
Wednesday 5 June 13

39. Present your dashboards.
Wednesday 5 June 13

40. Build Usable & Useful Visualisations
Step 5: How to evaluate visualisations?
Wednesday 5 June 13

41. Step 5: How to evaluate visualisations?
Not so easy: how to measure improved insights?
Typical HCI metrics don’t always work that well
•time required to learn the system
•time required to achieve a goal
•error rates
•retention of the use of the interface over time
Wednesday 5 June 13

42. Step 5: How to evaluate visualisations?
Evaluate the right thing
Munzner, 2009
Wednesday 5 June 13

43. FURTHER READINGS
• ATour through theVisualization Zoo, Jeffrey Heer, Michael
Bostock,Vadim Ogievetsky
• http://queue.acm.org/detail.cfm?id=1805128
• Interactive dynamics for visual analysis, a taxonomy of tools
that support the fluent and flexible use of visualizations, Jeffrey
Heer, Ben Schneiderman
• http://queue.acm.org/detail.cfm?id=2146416
Wednesday 5 June 13

44. POINTERS
• http://wearecolorblind.com/articles/quick-tips/
• http://infosthetics.com
• http://www.visualcomplexity.com/vc/
• http://bestario.org/research/remap
• ... (a lot more online! )
Wednesday 5 June 13

45. LIBRARIES
• D3.js
• http://www.jerryvermanen.nl/datajournalismlist/
• Processing
• http://wiki.okfn.org/OpenVisualisation
• http://flare.prefuse.org/
• http://iv.slis.indiana.edu/sw/
• http://abeautifulwww.com/2008/09/08/20-useful-visualization-libraries/
• Tableau software
• R
• Multitouch4J
• Manyeyes...
• ...
Wednesday 5 June 13

46. FURTHER READINGS
• “Readings in InformationVisualization: UsingVision toThink”,
Card, S et al
• “Now i see”,“Show Me the Numbers”, Few, S.
• “Beautiful Evidence”,Tufte, E.
• “InformationVisualization. Perception for design”,Ware, C.
• BeautifulVisualization: Looking at Data through the Eyes of
Experts (Theory in Practice): Julie Steele, Noah Iliinsky
Wednesday 5 June 13

47. SLIDE ACKNOWLEDGMENTS
• Jan Aerts
• Sven Charleer
• Stephen Few
Wednesday 5 June 13

48. THANKYOU FORYOUR
ATTENTION!
joris.klerkx@cs.kuleuven.be
@jkofmsk
48
Wednesday 5 June 13