This document summarizes learning analytics and information visualization tools developed at Tampere University of Technology. It discusses how these tools can monitor student activity and participation during online courses to help identify at-risk students, motivate participation, and inform pedagogical strategies. Specific tools are described that visualize student networks, connections between students and course content, keyword usage, and relationships between participation and academic performance. The tools provide benefits for both teachers and students by offering insights into learning processes and strategies for improving performance.

1. Learning Analytics and Information
Visualization
Monitoring students’ activity and participation during online course
MathGeAr and MetaMath, 26 – 27 June 2014 in Tampere
Tampere University of Technology
Department of Mathematics
Intelligent Information Systems Laboratory (IISLab)
Kirsi Kuosa, Development manager
Anne Tervakari, Juho Koro, Meri Kailanto, Jukka Paukkeri

2. Content of the presentation
• Learning Analytics
● Learning Analytics for TUT
Circle and Moodle
● Research articles and
references
2Intelligent Information Systems Laboratory (IISLab)

3. Learning analytics
● Learning analytics can be defined as collection of methods
for measuring, collecting, analyzing and reporting of data
about learners, their actions and contexts, which can be
utilized for understanding and optimizing learning (e.g.
LAK 2011.)
● Focus is on the learning process including an analysis of
the relationship between learner, content, teacher, and
institution (Long & Siemens 2011.)
3Intelligent Information Systems Laboratory (IISLab)

4. Learning Analytics – Why?
● Information about students' participation and activity is required
while the course is still in progress, for example, to
● motivate and activate students, and promote their ongoing
participation
● help identify students who may be at risk of failing, and
● help make decisions on pedagogical strategies, actions and
interventions.
● The rich information about students' actions is recorded and stored
automatically in the log data of learning environment. However,
● large amounts of data is difficult to utilize for developing
learning and teaching
– data mining techniques, analysing methods, and visualizations tools
are usually not developed for pedagogical purposes
– methods and tools may be cumbersome for use by teachers and
students.
4Intelligent Information Systems Laboratory (IISLab)

5. LEARNING ANALYTICS FOR
TUT CIRCLE AND MOODLE
Analysing and visualizing educational learning data

6. TUT Circle
● A social media enhanced learning environment developed by IISLab
of TUT. http://www.tut.fi/piiri
● Built on Drupal, an open source content management
framework.
● Contains all the basic functionalities of a modern social media web
service including possibilities
● to publish different types of content (wiki pages, blog posts,
forum messages, news, events)
● to form groups and friendships with each others
● to send private messages, chat, exchange opinions, create,
contribute and comment on contents, share resources within and
between the groups
● to control the visibility of their information and control the
access to the information.
6Intelligent Information Systems Laboratory (IISLab)

7. 7Intelligent Information Systems Laboratory (IISLab)
TUT Circle
Desktop layout
Mobile layout

8. An interactive learning analytics tool
● The tool implemented in TUT Circle (later on in Moodle) supports analysis
and visualization of log data from online learning environment in real time,
and offers several different visualizations representing information about
students’ behaviour during an online course.
● For example, interactive visualizations representing
● students’ activity and participation during the course in general, which
offers students the possibility to compare their own activity with those
of other students
● (social) networks based on information exchange among students
● connections between students' and learning materials
● relationships between students' active/passive participation and
academic performance
● how students use keywords describing the core content of the learning
material and terms that are relevant to the subject matter.
8Intelligent Information Systems Laboratory (IISLab)

9. An interactive visualization dashboard, part 1
9Intelligent Information Systems Laboratory (IISLab)
Time Unique viewers
Date
Views
The content can be
filtered according to
time, date, unique
viewers and views by
clicking and dragging
on charts.
In the histogram
presenting the
distribution of students’
participation, all peaks
are at the date of a
deadline. This indicates
a strong element of
procrastination.

10. An interactive visualization dashboard, part 2
10
Type of content Activity of students
X-axis: number of
content produced.
Y-axis: views of content
Size of bubble: size of
content produced
Green: Active
Red: Passive
For teacher: possibility
to make a comparison
between students'
activity and filter
content according to
an author.
For student:
possibility to compare
his/her own activity to
other students’ activity
and filter content
according to an author.
Intelligent Information Systems Laboratory (IISLab)
The content can be filtered according to an author or type
of content by clicking on the visualization.

11. Network based on information exchange
(comments) among students
11
Visualization presenting network
based on the students' comments on the
other students' messages and content.
Study programs:
Information technology
Inform. & knowl. man.
Industrial engineering
Communication engin.
Teacher
Intelligent Information Systems Laboratory (IISLab)

12. Network based on information exchange
(readings) among students
12
Visualization presenting network
based on students' readings of the
other students' content. The nodes
may be filtered by study program.
Study programs:
Information technology
Electrical engineering
Mechanical engineering
Inform. & knowl. man.
Electrical engineering
Industrial engineering
Communication engin.
Teacher
Intelligent Information Systems Laboratory (IISLab)

13. Participation in weekly assignments
13Intelligent Information Systems Laboratory (IISLab)
Comparison between discussion
forums (here: weekly assignments)
according to amount of content
produced (number of words: green
area=median, pink area=average)
Comparison between content produced by one student
(orange area) and content produced on average/median

14. Connections between students and
learning materials
14Intelligent Information Systems Laboratory (IISLab)
Students
Learning material available in TUT Circle
Discussion forums
Cited web sources
The visualization shows
connections between the student
(here: Petri) and
1. discussion forums he participated in
2. chapters of learning material he
cited in his messages
3. web sources he cited in his messages.
For student: Possibility to examine
his/her use of references, and
compare his/her own actions to other
students’ actions.
For teacher:
Also possibility to filter information
presented in visualization according
to grades.

15. Visualization based on content
analysis
Keyword added by the teacher
Most used words by the students
Blog messages
Discussion forums
Chapters of ebook
The visualization shows, for example,
have the students used the keyword
“verkkopalvelu” added by the teacher,
and where they have used it. Thus the
visualization helps a social navigation
among the students.
For student: Possibility to find
interesting information related to
the keyword from learning material
or from content produced by other
students, which supports peer
learning
For teacher: Support for quality
evaluation of the students
performance (i.e. helps teacher to
find out if the students use
keywords in meaningful way in
their works).

16. Content of the online learning material
16Intelligent Information Systems Laboratory (IISLab)
The visualization represented the
hierarchically structured content
of the online learning material.
Below the circle pack is a tag cloud
visualization that represents the
issues discussed in the learning
material.
When the user selects one or more
keywords from the tag cloud, the
chapters related to those words will
be emphasized in the circle pack.
The user can navigate to the
content by clicking the chapter
represented in the circle pack.

17. Relationship between academic
performance and participation
17Intelligent Information Systems Laboratory (IISLab)
Grade Words producedContent producedContent viewedContent read
Passive participation Active participation

18. Relationship between academic
performance and participation
18Intelligent Information Systems Laboratory (IISLab)
Grade Content read Content viewed Content produced Words produced
Passive participation Active participation

19. Relationship between academic
performance and participation
19Intelligent Information Systems Laboratory (IISLab)
For teacher: Possibility to
make a comparison between
students' participation and
academic performance, and
filter content according to
grade and student.
For student: Possibility to
compare his/her own
participation to other
students’ actions and
academic performance.

20. Summa summarum
● Learning analytics and visualization of learning data can produce valuable
information for both teachers and students. For example possibilities to
● verify student interaction and collaboration activities,
● see latent ties among students and learning materials or external web
resources
● observe the evolution of student active/passive participation
● investigate relationship between active/passive participation or
undesired behaviour like procrastination and academic performance.
● Visualizations based on content analysis can support to obtain overviews,
categorize, navigate, search for relevant content, and evaluate the quality
of the content.
20Intelligent Information Systems Laboratory (IISLab)
Please, see the video that gives an overview of the different
visualizations done at IISLab in 2009-2013 in a Finnish national
Campus Conexus project financed by the European Social Fund:
http://vimeo.com/84297046

21. Benefits for the teachers
● Help to evaluate the quality of a course’s instructional design from
viewpoint of pedagogical usability (Silius & Tervakari 2003,2007).
● Provide useful information about students’ learning types and
activity, and evolution of the participation.
● Help to identify undesired behaviour such as procrastination, and
students who might have risk of failing or dropping out. à
Possibilities to make strategic interventions or provide guidance just
in time.
● Support comparison of activity and active/passive participation
among the students, and the quality evaluation of the students’
performance. à Support for fair and just assessment of the
students’ learning performance.
● Help the to maintain the quality of the learning material, for
example, by helping to identify the least used parts of the material.
21Intelligent Information Systems Laboratory (IISLab)

22. Benefits for the students
• Helps to monitor and evaluate own performance processes
and learning outcomes, and compare own activity levels
with those of other students. à Support strategic
adjustments for improving own performance.
• Helps to find new and interesting references to utilize in
own works.
• Visualization based on content analysis can help to see the
overview of the actual topics discussed, reflect the quality
of own content, and support educational knowledge
discovery.
• May increase motivation (especially among the students
who have a competitive spirit).
22Intelligent Information Systems Laboratory (IISLab)

23. Research articles 1/3
● Kuosa, K., Koro, J., Tervakari, A-M., Paukkeri, J. & Kailanto, M. 2014. Content analysis and
visualizations, Tools for social enhanced learning environment. 17th International Conference
on Interactive Collaborative Learning (ICL 2014), 3 – 6 December 2014 in Dubai. Accepted for
publication.
● Silius, K., Kailanto, M. & Tervakari, A-M. 2011. Evaluating the quality of the social media in an
educational context. International Journal of Emerging Technologies in Learning, vol. 6, issue
3, 21 -27. doi: http://dx.doi.org/ijet.v6i3.1732
● Silius, K. & Miilumäki, T. 2009. Students’ Motivations for Social Media Enhanced Studying and
Learning. In: Proceedings of International Technology Enhanced Learning Conference 2009
(TELearn 2009) on the 6-8 October 2009 Taipei, Taiwan. CD-ROM.
● Silius, K., Miilumäki, T., Huhtamäki, J., Tebest, T., Meriläinen, J. & Pohjolainen, S. 2010.
Social Media Enhanced Studying and Learning in Higher Education. In Education Engineering
(EDUCON), 2010 IEEE, Conference Proceedings, 14-16 April 2010, Madrid, Spain, pp.
137-143.doi: http://dx.doi.org/10.1109/EDUCON.2010.5492586
● Silius, K., Miilumäki, T., Huhtamäki, J., Tebest, T., Meriläinen, J. & Pohjolainen, S. 2010.
Students’ Motivations for Social Media Enhanced Studying and Learning. In Knowledge
Management & E-Learning: An International Journal (KM&EL), Vol. 2, No. 1, pp. 51-67.
Available at
http://www.kmel-journal.org/ojs/index.php/online-publication/article/view/55/39 .
23Intelligent Information Systems Laboratory (IISLab)

24. Research articles 2/3
● Silius, K., Tervakari, A-M. & Kailanto, K. 2013. Visualizations of user data in social media
enhanced web-based environment in higher education. In: Proceedings of Global Engineering
Education Conference (EDUCON), Synergy from Classic and Future Engineering Education,
2013 IEEE, the 13-15 March 2013 in Berlin, Germany. Extended version published in
International Journal of Emerging Technologies in Learning, vol. 8, Special issue. DOI:
http://dx.doi.org/10.3991%2Fijet.v8iS2.2740
● Silius, K., Tervakari, A-M., Huhtamäki, J., Tebest, T., Marttila, J., Kailanto, M. & Miilumäki,
T. 2011. Programming of Hypermedia – Course Implementation in Social Media. In:
Proceedings of the 2011 2nd International Congress on Computer Applications and
Computational Science. Advances in Intelligent and Soft Computing, 2012, Volume 144/2012,
369-376. Springer Berlin/Heidelberg. DOI: http://dx.doi.org/10.1007/978-3-642-28314-7_50
● Silius, K., Tervakari, A-M., Kailanto, M., Huhtamäki, J., Marttila, J., Tebest, T. & Miilumäki,
T. 2011. Developing an Online Publication – Collaborating among Students in Different
Disciplines. In: Proceedings of the 2011 2nd International Congress on Computer Applications
and Computational Science. Advances in Intelligent and Soft Computing, 2012, Volume
144/2012, 361-367. Springer Berlin/Heidelberg. DOI:
http://dx.doi.org/10.1007/978-3-642-28314-7_49
● Tervakari, A-M., Kuosa, K., Koro, J., Paukkeri, J. & Kailanto, M. 2014. Teacher’s learning
analytics tools in social media enhanced learning environment. 17th International Conference
on Interactive Collaborative Learning (ICL 2014), 3 – 6 December 2014 in Dubai. Accepted for
publication.
24Intelligent Information Systems Laboratory (IISLab)

25. Research articles 3/3
● Tervakari, A-M., Marttila, J., Kailanto, M., Huhtamäki, J., Koro, J. & Silius, K. 2013.
Developing Learning Analytics for TUT Circle. In: Ley, T., Ruohonen, M., Laanpere, M. &
Tatnall, A. (eds.). Open and Social Technologies for Networked Learning. IFIP Advances in
Information and Communication Technology. Springer Berlin/Heidelberg, 101 – 110.
http://dx.doi.org/10.1007/978-3-642-37285-8_11
● Tervakari, A-M., Silius, K. & Kailanto, K. 2013. Students’ Participation in a Social Media
Enhanced Learning Environment. In: Proceedings of Global Engineering Education Conference
(EDUCON), Synergy from Classic and Future Engineering Education, 2013 IEEE, the 13-15
March 2013 in Berlin, Germany. Extended version is published in International Journal of
Emerging Technologies in Learning, vol. 8, Special issue. DOI:
http://dx.doi.org/10.3991%2Fijet.v8iS2.2740
● Tervakari, A-M., Silius, K., Koro, J., Paukkeri, J., and Pirttilä, O. 2014. Usefulness of
information visualizations based on educational data. In Proceedings ofn the 4th IEEE Global
Engineering Education Conference (EDUCON 2014) on 2nd – 6th of April 2014 in Istanbul,
Turkey, 142 - 151. doi: http://dx.doi.org/10.1109/EDUCON.2014.6826081
● Tervakari, A-M., Silius, K., Tebest, T. Marttila, J., Kailanto, M, & Huhtamäki, J. 2012. Peer
learning in Social Media Enhanced Learning. International Journal of Emerging Technologies in
Learning, vol. 7, issue 3, 35-42. http://dx.doi.org/10.3991/ijet.v7i3.2173
25Intelligent Information Systems Laboratory (IISLab)

26. References
• LAK 2011. 1st International Conference on Learning Analytics and
Knowledge, Banff, Alberta, 27 Feb – 1 March, 2011,
https://tekri.athabascau.ca/analytics/
• Long, P, and Siemens, G. 2011. Penetrating the Fog: Analytics in Learning
and Education. EDUCAUSE Review, vol. 46, no. 5, 30 – 40.
https://net.educause.edu/ir/library/pdf/ERM1151.pdf
• Silius, K. & Tervakari, A-M. 2003. An Evaluation of the Usefulness of Web
based Learning Environments, The Evaluation Tool into the Portal of
Finnish Virtual University,” in: Pearrocha, V. & alt. (ed.) mENU 2003 – Int.
Conf. on University Networks and E-learning 2003, 8-9 May 2003 in
Valencia, Spain. Proc. of mENU.
• Silius, K. & Tervakari, A-M. 2007. Variety of Quality Experiences on Web-
Based Courses. In: Spector, J. M. et al. (eds.). Proc. of the 7th IEEE
Int.Conference on Advanced Learning Technologies ICALT 2007, July 18-20
2007, Niigata, Japan, 858-86. DOI:
http://dx.doi.org/10.1109/ICALT.2007.278

27. Thank you for your attention!
Questions?
27Intelligent Information Systems Laboratory (IISLab)
Photo: Mika Hirsimäki

28. Campus Conexus (2009-04/2014)
• This study is part of a Finnish
national project called Campus
Conexus.
• Purpose is to strengthen the
cultural practices of five Finnish
universities and to promote
learning and teaching.
• Aim is to study how to engage
students into university studies by
enriching learning experiences for
example with online communities.
• Financed by European Social
Fund.