Towards Visual Analytics to Improve Scientific Reading & Writing
1. TOWARDS
VISUAL ANALYTICS
TO IMPROVE
SCIENTIFIC
READING & WRITING
Duygu Simsek
PhD Research Student
Knowledge Media Institute
The Open University, UK
The Open
Science
Laboratory
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Supervisors: Prof. Simon Buckingham Shum, Dr. Rebecca Ferguson, Dr. Anna De Liddo
Xerox Research Centre Europe: Ágnes Sándor
people.kmi.open.ac.uk/simsek
duygu.simsek@open.ac.uk
simsekduygu_
2. “Piled Higher and Deeper” by Jorge Cham www.phdcomics.com
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
3. Imagine if a machine that could detect…
Human analyst
-Highlight key contributions of the project outcomes
http://bit.ly/HumanVsXIP
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
5. Reading a Paper Writing a Paper
CRITICAL MIND
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Engaging with the literature
6. Being able to
critique and
contrast
Identifying
gaps in
our
knowledge
Identifying
when
significant
claims
made
Reading a Paper Writing a Paper
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Engaging with the literature
7. o What’s the research problem about X?
o What are the open questions about X?
o What has been found about X?
o What are the research themes for X?
o Are there any major advances concerning X?
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Scholarly writing has particular hallmarks!
8. oMeta-discourse Signals
o The problem, the claim, the
argument, the evidence and the
implications
o Explicitly organises the
discourse, expresses a
viewpoint, engages the readers,
and signals the writer's stance
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Meta-discourse
9. oAutomatic processing of scientific documents
oRecognition of the rhetorically significant sentences
o8 categories of Rhetorical Moves
o Background Knowledge
o Summarising
o Tendency
o Novelty
o Significance
o Surprise
o Open Question
o Contrasting Ideas
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Xerox Incremental Parser (XIP)
10. Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Xerox Incremental Parser (XIP)
11. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
12. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
13. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
TENDENCY:
... emerging as a promising
approach
Our understanding ... has
grown exponentially ...
Growing recognition of the
importance ...
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
14. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
NOVELTY:
New insights provide direct
evidence…
…suggest a new approach…
Results define a novel role ...
TENDENCY:
... emerging as a promising
approach
Our understanding ... has
grown exponentially ...
Growing recognition of the
importance ...
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
15. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
NOVELTY:
New insights provide direct
evidence…
…suggest a new approach…
Results define a novel role ...
TENDENCY:
... emerging as a promising
approach
Our understanding ... has
grown exponentially ...
Growing recognition of the
importance ...
SIGNIFICANCE:
studies ... have provided important
advances
... is crucial for ... understanding
valuable information ... from
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
16. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
NOVELTY:
New insights provide direct
evidence…
…suggest a new approach…
Results define a novel role ...
TENDENCY:
... emerging as a promising
approach
Our understanding ... has
grown exponentially ...
Growing recognition of the
importance ...
SIGNIFICANCE:
studies ... have provided important
advances
... is crucial for ... understanding
valuable information ... from
SURPRISE:
We have recently observed ...
surprisingly
We have identified ... unusual
The recent discovery ... suggests
intriguing roles
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
17. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
NOVELTY:
New insights provide direct
evidence…
…suggest a new approach…
Results define a novel role ...
OPEN QUESTION:
Little is known …
… role … has been elusive
Current data is insufficient…
TENDENCY:
... emerging as a promising
approach
Our understanding ... has
grown exponentially ...
Growing recognition of the
importance ...
SIGNIFICANCE:
studies ... have provided important
advances
... is crucial for ... understanding
valuable information ... from
SURPRISE:
We have recently observed ...
surprisingly
We have identified ... unusual
The recent discovery ... suggests
intriguing roles
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
18. Xerox Incremental Parser (XIP)
BACKGROUND
KNOWLEDGE:
Recent studies indicate …
the previously proposed …
… is universally accepted
NOVELTY:
New insights provide direct
evidence…
…suggest a new approach…
Results define a novel role ...
OPEN QUESTION:
Little is known …
… role … has been elusive
Current data is insufficient…
TENDENCY:
... emerging as a promising
approach
Our understanding ... has
grown exponentially ...
Growing recognition of the
importance ...
CONTRASTING IDEAS:
In contrast with previous
hypotheses ...
... inconsistent with past
findings ...
SIGNIFICANCE:
studies ... have provided important
advances
... is crucial for ... understanding
valuable information ... from
SURPRISE:
We have recently observed ...
surprisingly
We have identified ... unusual
The recent discovery ... suggests
intriguing roles
SUMMARISING:
The goal of this study ...
Here, we show ...
Our results ... indicate
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
19. o Fine for researchers or
machines but it is not
learner/educator friendly
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
XIP Output
20. Visual Analytics of XIP Output
o Dataset
Proceedings of the Learning Analytics and Knowledge (LAK)
conferences since 2011and a journal special issue, and the
Educational Data Mining (EDM) conferences since 2008 and
journal.
o XIP Analysis
All of the papers (66 LAK and 239 EDM papers 305 in
total) were analysed using XIP, extracting 7847 sentences and
40163 concepts.
o Paper Prototype
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
21. o Similar patterns in LAK &
EDM
o Summary & Contrast
categories higher &
rising
Visual Analytics of XIP Output
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
22. Visual Analytics of XIP Output
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
23. Visual Analytics of XIP Output
http://bit.ly/DCLA13
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
24. Learning Analytics Potential?
oA promising language technology
oNow visual analytics built on top of it
oWhat happens when we bring it into the realm
of learning?
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
25. S288 Practical Science
o ~300 undergraduate students
o 5 strands:
o Physics and Astronomy
o Chemistry and Analysis
o Environmental Sciences
o Earth and Environment
o Biology and Health
o Collaborative scientific writing assignment
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Learning Analytics Potential?
26. Learning Analytics Potential?
XIP Highlighted Student Essays
Is the use of XIP’s rhetorical
categories predictive of final
grades?
Student Grades
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
27. Learning Analytics Potential?
What is the overlap between
XIP’s output and how tutors
judge quality?
Tutor Highlighted Student Essays
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
XIP Highlighted Student Essays
28. Do learners find it helpful to receive
formative feedback of this sort on their
own writing, or of the literature?
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013
Learning Analytics Potential?
C
O
L
L
A
B
O
R
A
T
I
V
E
W
R
I
T
I
N
G
E
D
I
T
O
R
29. Resources & Contact Details
o Simsek, D., Buckingham Shum, S., Sándor, Á., De Liddo, A. and Ferguson, R.
(2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of
Scientific Metadiscourse. 1st International Workshop on Discourse-Centric
Learning Analytics. (3rd International Conference on Learning Analytics &
Knowledge, 8 April 2013, Leuven, Belgium). http://bit.ly/DCLA13
o Taibi, D., Sándor, Á., Simsek, D., Buckingham Shum, S., De Liddo, A. and
Ferguson, R. (2013) Visualizing the LAK/EDM Literature Using Combined
Concept and Rhetorical Sentence Extraction, 1st Learning Analytics and
Knowledge Data Challenge at Learning Analytics and Knowledge (LAK ’13),
Leuven, Belgium http://bit.ly/DataChal
o De Liddo, Anna; Sándor, Ágnes and Buckingham Shum, Simon (2012).
Contested Collective Intelligence: rationale, technologies, and a human-machine
annotation study. Computer Supported Cooperative Work (CSCW), 21(4-5) pp.
417–448. http://bit.ly/HumanVsXIP
people.kmi.open.ac.uk/simsek
duygu.simsek@open.ac.uk
simsekduygu_
Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013