A Survey of Annotation Tools for Lecture Materials
1. A Survey of Annotation Tools
For Lecture Materials
‣ Kai Michael Höver
‣ Guido Rößling
Department of Computer Science
Technische Universität Darmstadt
Germany
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver |
Freitag, 22. Oktober 2010
2. Introduction
Lecture (Recording) Systems
present learning material
annotate learning material
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 2
Freitag, 22. Oktober 2010
3. Active annotating & interaction
Studies have shown the importance of active learning rather than
passively receiving information
Making annotations can support students in structuring information and to
enhance them with their own ideas and knowledge
Interaction is also an important element of teaching & learning
Interaction can be achieved by exchanging these annotations
Intentions:
highlighting and directing attention
sharing information
asking questions ...
Stimulus for discussions
Hake, R., 1998, Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, American Journal of Physics, 66(1), pp. 64-74.
Koivunen, M. R. & Swick, R. (2001). Metadata based annotation infrastructure offers flexibility and extensibility for collaborative applications and beyond. In Proceedings of the first international
conference on knowledge capture (KCAP 2001), victoria, BC, canada
McConnell, J. J. (1996). Active learning and its use in computer science. In ITiCSE '96: Proceedings of the 1st conference on integrating technology into computer science education. ACM.
Prince, M. (2004). Does active learning work? A review of the research. A Review of the Research. Journal of Engineering Education, 93(3), 223-231.
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 3
Freitag, 22. Oktober 2010
4. Many systems exist!
Which to choose? What‘s missing?
AWS StuPad
AOF Ubiquitous
Classroom Presenter
Presenter WeLCoME
CoScribe WriteOn
DEBBIE ...
DyKnow
eMargo
Livenotes
MRAS
PaperCP
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 4
Freitag, 22. Oktober 2010
5. What‘s coming next?
Short presentation of 14 systems (no claims of being complete,
but interesting :-)
We cover
Systems that enable students to annotate learning material
scientific papers available
MS PowerPoint, OpenOffice, Keynote
Annotea
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 5
Freitag, 22. Oktober 2010
6. Taxonomy
Scenarios
Annotation Material
Annotation Classes
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 6
Freitag, 22. Oktober 2010
7. Taxonomy: Scenarios
Educator can annotate during/after a lecture
Student can annotate during/after a lecture
Student can access educator‘s annotations during/after a lecture
Student can access fellow student‘s annotations during/after lecture
Educator can access student‘s annotations during/after a lecture
Educator can select among student‘s annotations, add comments,
and present both to other students
Student presents an application from his/her laptop on a public
screen; the educator or students can add annotations
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 7
Freitag, 22. Oktober 2010
8. Taxonomy: Annotation Material
Slide (electronic or printed)
Whiteboard or blank sheet
Background (templates like graph paper, sheet of music)
Lecture notes, text document
Video
Software application
Annotation
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 8
Freitag, 22. Oktober 2010
9. Taxonomy: Annotation Classes
Text notes (with/without hyperlink support)
Digital ink / ink
Audio
Tags / Bookmarks
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 9
Freitag, 22. Oktober 2010
10. eClass (Classroom 2000) + StuPad
eClass integrates captured audio
and video with slides and
instructor‘s annotations
StuPad‘s motivation: desire of
students to add private
annotations
➡ blank notebook (not sharable)
Abowd, G. D. (1999). Classroom 2000: An experiment with the instrumentation of a living educational environment. IBM Syst. J., 38(4), 508-530.
Truong, K. N. & Abowd, G. D. (1999). Stupad: Integrating student notes with class lectures. In CHI '99 extended abstracts on human factors in computing systems. ACM.
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 10
Freitag, 22. Oktober 2010
11. Authoring On the Fly (AOF)
lecture recording system
educator can annotate the slides
transmits both slides and educator‘s annotations to students‘ PCs
students can add their own annotations on a different layer
!
Müller, R. & Ottmann, T. (2000). The "authoring on the fly" system for automated recording and replay of (tele)presentations. Multimedia Systems, 8(3), 158-176.
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 11
Freitag, 22. Oktober 2010
13. Classroom Presenter
C. Liao et al.
Classroom/Ubiquitous Presenter PaperCP
d Tablet PCs. Instructor
the digital solu-
vantages in data
and archiving,
drawbacks are
d with these
a degraded
and writing ex-
due to limited Students
ize and screen Fig. 1. (Left) The original Tablet PC interface of a digital Active
Public display Learning infrastructure. (Right) The new equivalent interface,
n, the cost of the
PaperCP, based on Anoto technology, which consists of Blue-
Anderson, R. & Linnell, N. (2009). Promoting interaction in distance education. Elearn, 2009(8).
andownthe limita- shown here, the instructor presents a slide with anprintouts.
Figure 1. Classroom Presenter.The system runs on instructor, student, and public displays, each
Wilkerson, M., Griswold, W. G., & Simon, B. (2005). Ubiquitous presenter: Increasing student access and control in a digital lecturing environment. In SIGCSE '05. ACM.
with its user interface. In the scenario
tooth digital pens and
Liao, C. et al. (2007). Papercp: Exploring the integration of physical and digital affordances for active learning. In Human-Computer interaction – INTERACT 2007.
activity.The students write solutions to the activity on their Tablet PCs and submit their
posed thethe instructor view) and thenComputerstudent solutions in aon the public | Kai Michael Höver | Slide 13
answers to by battery can preview the
instructor.The instructor
Oct 22, 2010 | Department of
on the right of
film strip (shown
Science | Telecooperation
selectively show student answers
easefor class discussion.
display the tension
Freitag, 22. Oktober 2010
14. !"#$%&&'$$ $$()(*+,-$*./01234516$7$"869$,:;28<;$
!"#$%&'()*+, -"%, ./#'0, !%)%, (, )%1()2(&/%, *%3#4%, 5'), $"%#),
Livenotes
$#1%, 67889:;, &<$, !%)%, /#1#$%*, #=, $%)10, '5, >)'4%00#=?,
!
4(>(&#/#$@, 67A9, BCD, EFG., .HI:, (=*, <0(&#/#$@, 64)(1>%*,
!
2%@&'()*;, /#1#$%*, 04)%%=, 0%=0#$#3#$@, $', >%=, #=><$;, (=*, (,
!
>)%00<)%J0%=0#$#3%,04)%%=,$"($,$)%($%*,(44#*%=$(/,"(=*,4'=$(4$,
!#$",$"%,04)%%=,(0,>%=,#=><$:+,,
!
!
!
supports real-time conversation !
within small groups of students ! !
during lectures !
!
shared whiteboard for students !
!
instructor can load PPT file into the !
whiteboard‘s background !
!
!
!
! ,
! !"#$%&'()'*+&',"-&./0&1'$1&%'".0&%234&'2/%'0+&'&56&%"7&.0'
! 8&14%"9&8'".'0+"1'636&%:';"0+'0+&'"76/%0&8'934<#%/$.8'1="8&'
/44$6>".#'0+&'0/6'=&20?+3.8'3%&3'/2'0+&';+"0&9/3%8)'
!
G%4'=*;,$"%,#=$%)5(4%,!(0,'3%)"(</%*,#=,KLLK,$',#=4')>')($%,
!
=%!, ?)'<>, (!()%=%00, (=*, $', %=(&/%, /%4$<)%, 0/#*%0, $', &%,
!
Kam, M., Wang, J., Iles, A., Tse, E., Chiu, J., Glaser, D., et al. (2005). Livenotes: A system for cooperative and augmented note-taking in lectures. '5, $"%, ./#'0, (=*, 1(*%, #$,
#1>')$%*+, -"#0, $(M%*, $"%, 4(>(4#$@,
In CHI '05: Proceedings of the SIGCHI conference on human factors in computing systems. ACM !
#1>'00#&/%, $', #=$)'*<4%, N#3%='$%0, #=$', (=, <=*%)?)(*<($%,
!
4/(00,'=,(,='=J$)#3#(/,04(/%,!#$"'<$,*#0)<>$#'=+,O%,$"%)%5')%,
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 14
!
1#?)($%*, N#3%='$%0, $', /(>$'>0, (=*, 5'<=*, $"($, #$, !')2%*,
%55%4$#3%/@, !#$", 0#?=#5#4(=$/@, ?)%($%), 4'1><$#=?, >'!%)+,
!
Freitag, 22. Oktober 2010 -"#)*;, $"%, #=$%)5(4%, !(0, 4"(=?%*, #=, KLLKJKLLP, $', #=4/<*%,
15. DEBBIE, DyKnow
L6:;&)--0A--M*&6%;+-!%,'),'-D8=)+-H@*67*>7)-6,-B8C,%?
3,-'()-%'()&-(*,45-6.-'()-')*/()&-%&*778-4).6,)4-'&))-')&96,%7%:8-+;/(-
DePauw Electronic Black Board for Interactive
<=*&),'<5-*,4-<+6>76,:5<-'()-+';4),'+-96:('-/(%%+)-'%-?&6')-%&-'8=)-'()+)-4).6,
'()6&-=&6@*')-,%')+-9*&:6,A-!"#$%&'('+(%?+-'()-B8C,%?-6,')&.*/)A-D()-'(6/E-6
Education
Note annotation in electronic classrooms 87
7).'F(*,4-=*,)-?*+-4&*?,->8-'()-')*/()&-*,4-'()-'(6,,)&-6,E-?*+-*44)4->8-*-+';4
&6:('-=*,)-+(%?+-'()-+';4),'G+-=&6@*')-,%')-*&)*A-H+-)I=7*6,)4-9%&)-.;778-6,-+)/
*'-'()-),4-%.-/7*++-+';4),'+-/*,-=&6,'-%&-+*@)-'()6&-,%')>%%E-.%&-7*')&-&)@6)?-*,
!!"!#$%&'()*+')&,-!%,.)&),/)
Figure 8. Screen dump of participant’s version of DEBBIE.
L6:;&)--0A--M*&6%;+-!%,'),'-D8=)+-H@*67*>7)-6,-B8C,%? 6. User experiences, conclusions, and future work
3,-'()-%'()&-(*,45-6.-'()-')*/()&-%&*778-4).6,)4-'&))-')&96,%7%:8-+;/(-*+-<&%%'5 to the system has been positive, although evidence to date is only
Reaction
),'<5-*,4-<+6>76,:5<-'()-+';4),'+-96:('-/(%%+)-'%-?&6')-%&-'8=)-'()+)-4).6,6'6%,+-6,
anecdotal. For example, one of the authors is currently teaching a ‘Theory
&-=&6@*')-,%')+-9*&:6,A-!"#$%&'('+(%?+-'()-B8C,%?-6,')&.*/)A-D()-'(6/E-6,E-6,-'()
of Computation’ class which is being completely delivered using the system.
The course has an enrollment of 13 students, and each student has access to
(*,4-=*,)-?*+-4&*?,->8-'()-')*/()&-*,4-'()-'(6,,)&-6,E-?*+-*44)4->8-*-+';4),'A--D()
'-=*,)-+(%?+-'()-+';4),'G+-=&6@*')-,%')-*&)*A-H+-)I=7*6,)4-9%&)-.;778-6,-+)/'6%,-0AJ5 tablet to use large lecture classes usingbeen structured so that each ICLS'08: Proceedings of the 8th international
a video
in
during class. The course has
L6:;&)-NA-B8C,%?-!%77*>%&*'6@)-$%')-D*E6,:-O,')&.*/)
Johri, A. & Lohani, V. K. (2008). Creating a participatory learning environmentan ‘engagement grade’ which is based on the computing. In
pen-based quality of the
student receives
)-),4-%.-/7*++-+';4),'+-/*,-=&6,'-%&-+*@)-'()6&-,%')>%%E-.%&-7*')&-&)@6)?-*,4-+';48A- Learning Sciences.
conference for the learning sciences. International Society of the annotations which are added to the electronic notebooks, and on the quality
Berque, D. (2006). An evaluation of a broad deployment of dyknow software to support note taking and interaction distributed through computers. J. Comput. Small Coll., 21(6), 204-216.
of the answers given to short in-class quizzes which are using pen-based
B8C,%?-')*/()&+-%.'),-+(*&)-=&)=*&)4-/%,'),'-?6'(-+';4),'+-.%77%?6,:-*
Berque, D., Bonebright, T., & Whitesell, M. (2004). Using pen-based computers across approximately every 10 to 15 minutes. SIGCSE Bull., 36(1), 61-65.
the electronic notebooks the computer science curriculum. The idea of
=&*/'6/)5- *,4- 46+/7%+)<- =*'')&,- '(*'- ?)- /*77- =&%:&)++6@)- 46+/7%+;&)A- - D(6+- ')
these quizzes is to require the students to be engaged with the material in an
ongoing way. When asked to provide anonymous written assessment of the use
Oct 22, 2010 | Department of Computer ),/%;&*:)+-+';4),'+-'%-/%,'&6>;')-'%-'()-+(*&)4-*&'6.*/'-,%')>%%E5-'()&)>8-()7=
Science DEBBIE in this class, students generally reacted favourably. One student,
of | Telecooperation | Kai Michael Höver | Slide 15
referring to the frequent electronic quizzes, wrote ‘. . .giving us time to work and
'%-'(6,E-*>%;'-/%,'),'-*+-6'-6+->)6,:-=&)+),')4A--L%&-)I*9=7)5-*-')*/()&-96:('-=&%
think about topics is the only way we are going to understand. . .’ while another
wrote ‘. . .it is nice to be able to take notes on top of your notes for the class and
Freitag, 22. Oktober 2010 /7*++-?6'(-*-B8C,%?-=*:)-'(*'-=%+)+-*-+(%&'-=&%:&*996,:-=&%>7)9A---D()-')*/(
I realize that I am getting the main points of the lecture as opposed to picking
16. WriteOn 1.0
Instructor can annotate running
Windows applications
broadcast of defined area &
annotations to students‘ PCs
Students can add ink notes
Lecturer can add annotations to
the recording
!
Chandrasekar, S., Tront, J. G., & Prey, J. C. (2009). Writeon1.0: A tablet pc-based tool for effective classroom instruction. In Iticse '09: Proceedings of the 14th annual ACM SIGCSE
conference on innovation and technology in computer science education. ACM.
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 16
Freitag, 22. Oktober 2010
17. MRAS
Microsoft Research Annotation System
annotating lecture videos
personal notes & discussions
text or audio
linking of Web resources (URLs)
Figure 2: Web-based MRAS interface.
Bargeron, D., Grudin, J., Gupta, A., Sanocki, E., Li, F., & Leetiernan, S. (2002). Asynchronous collaboration around and Functions
4.4 Annotation System Features multimedia applied to on-demand education.
Journal of Management Information Systems, 18(4), 117-145. The video in the upper left of the browser window in Figure 2 is displayed with a standard media player. The
slides on the right are synchronized with the video. In the lower left is an annotation set: comments, questions, and
replies made by previous viewers of the video. Each annotation is linked to the specific point in the video at which
Oct 22, 2010 | Department of Computer Science | Telecooperation |paused when the video; the reply|below itThealso highlighted. To the annotation thatthe annotation preview
the video
Kai Michael Höver created. is red arrow points to the lower right is was created closest to
the current position of
the annotation was
Slide 17
window, which automatically shows text associated with the currently highlighted annotation.
As the video plays, annotation titles scroll by. A viewer can click on an annotation and read it in the preview
Freitag, 22. Oktober 2010 frame, right-click to seek to the point in the video where an annotation was created, or reply to it, forming a threaded
18. We-LCoME
Wiki e-Learning Compound
Multimedia Environment
combines Wiki-functionality
(DokuWiki) with enriching Lecture
Recording
Users add additional information
(e.g. captions) in the Wiki (special
syntax)
Additional information are aggregate
with the recording to a SMIL video
lecture
Ferretti, S., Mirri, S., Muratori, L. A., Roccetti, M., & Salomoni, P. (2008). E-Learning 2.0: You are we-lcome!.
W4A '08: Proceedings of the 2008 international cross-disciplinary conference on web accessibility (W4A). ACM.
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 18
Freitag, 22. Oktober 2010
19. eMargo
digital margin for slides and lecture
notes/text documents
highlighting
discussion support
tagging
Sesink, W., Göller, S., Rößling, G., & Hofmann, D. (2007). Emargo: Eine digitale randspalte zum selbststudium (nicht nur) der informatik. In Proceedings der pre-conference
workshops der 5. E-Learning fachtagung informatik (delfi 2007). Logos Verlag Berlin.
Rößling, G., Joy, M., Moreno, A., Radenski, A., Malmi, L., Kerren, A., et al. (2008). Enhancing learning management systems to better support computer science education.
SIGCSE Bull., 40(4), 142-166
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 19
Freitag, 22. Oktober 2010
20. CoScribe
annotation system of handwritten
notes for both printed and digital
slides
synchronization from printed
slides to digital slides
annotations can be categorized
(e.g. important, question, ...)
annotations can be shared
!
Steimle, J., Brdiczka, O., & Mühlhäuser, M. (2009). Coscribe: Integrating paper and digital documents for collaborative knowledge work. IEEE Trans. Learn. Technol., 2(3), 174-188.
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 20
Freitag, 22. Oktober 2010
21. Taxonomy: Scenarios (revisited)
Educator can annotate during/after a lecture: 6/4, 3 both, 1 not
Student can annotate during/after a lecture: 8/4, 2 both
Student can access educator‘s annotations during/after a lecture:
4/4 , 5 both, 1 not
Student can access fellow student‘s annotations during/after lecture:
1/4, 2 both, 7 not
Educator can access student‘s annotations during/after a lecture:
3/6, 2 both, 3 not
Educator can select among student‘s annotations, comment, and present them
to other students: Yes = 3
Student presents an application from his/her laptop on a public screen; the
educator or students can add annotations: Yes = 0
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 21
Freitag, 22. Oktober 2010
22. Taxonomy: Annotation Material
(revisited)
Slide (electronic or printed)
4 no slides, 8 electronic slides, 1 printed slides, 1 both
Whiteboard or blank sheet: 7 yes, 7 no
Background (templates like graph paper, sheet of music)
2 yes, 12 no
Lecture notes, text documents: 2 yes, 12 no
Video: 2 yes, 12 no
Application: 2 yes, 12 no
Annotations: 3 yes, 11 no
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 22
Freitag, 22. Oktober 2010
23. Taxonomy: Annotation Classes
(revisited)
Text notes (with/without hyperlink support)
10 yes (2 supporting hyperlinks)
Digital ink / ink
9 digital ink, 2 pen (Anoto)
Audio: 1
Tag / Bookmark
1 tags, 2 bookmarks
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 23
Freitag, 22. Oktober 2010
24. Summary
Annotation material
Slides only?
text documents? Web sites?
Annotation classes
digital ink & text annotations
integration of Web resources? Web 2.0 content?
Focus on one specific period of time
during/after a lecture
sharing of annotations only during a lecture
less support of student interaction
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 24
Freitag, 22. Oktober 2010
25. Thank you
for listening!
Questions?
Oct 22, 2010 | Department of Computer Science | Telecooperation | Kai Michael Höver | Slide 25
Freitag, 22. Oktober 2010