2021_03_26 "Eye-tracking techniques and methods in e-learning environments" - Aleksandra Klasnja Milicevic

1. Eye-tracking Techniques and
Methods in e-Learning
Environments
Aleksandra Klasnja-Milicevic
Department of Mathematics and Informatics,
Faculty of Sciences, University of Novi Sad, Serbia

2. Agenda
The human eye and eye-tracking device
1
Eye-tracker: possible usages
2
Eye-tracking technology in digital learning
environments
3
Assessing Learning Styles Through Eye Tracking
4

3. Eye-tracking technology
Eye-tracking technology - a set of methods and techniques used to: discover, identify
and record the activities of eye movements.
Eye-tracking system confirmed its usefulness in terms of:
• identifying behavioural response,
• presenting cognitive load,
• providing an alternative means for human-computer interaction,
• prompting interface design and
• adapting appearance of elements according to user data.
Incorporating eye tracking into adaptive e-learning systems - useful in a process of
adaptation to the requirements and needs of the learner.
3

4. Structure of the Human Eye
https://scienceeasylearning.wordpress.com/2015/05/27/structure-of-human-eye-and-its-working-and-defects-in-human-eye/
• Rays of light enter the cornea and through
the pupil are focused on the lens.
• From the lens, through the vitreous humor
the light is transmitted to the retina.
• Perceived by the photoreceptors of the
retina, the light is delivered to the brain by
means of the optical nerve.
4

5. There are two type of photoreceptors located in the fovea:
1. Cons:
• much less sensitive to light than rods
• do not saturate
• can discriminate colors (red, green and blue)
2. Rods:
• Work at very low levels of light.
• We use these for night vision.
• Don't help with color vision.
Structure of the Human Eye
5
https://scienceeasylearning.wordpress.com/2015/05/27/structure-of-human-eye-and-its-working-and-defects-in-human-eye/

6. 6
https://askabiologist.asu.edu/rods-and-cones

7. • The history of eye tracking started at the end of 1800’s;
• in 1879 Louis Èmile Javal observed that reading does not involve
a smooth sweeping of the eyes along the text, as previously
assumed, but a series of short stops (called fixations) and quick
saccades. This observation raised important questions about
reading, questions which were explored during the 1900s:
• theorized the existence fixations and saccades giving rise to
some questions:
• What are the words where the eyes stop?
• For how long?
• When does the gaze regress back to already seen words?
Eye tracking: a short history and definitions

8. • Eye tracking is the process of measuring the motion of the eye
relative to the head position.
The device used to make these measures is called eye tracker.
• A fixation occurs when a person looks at a specific point for a
time longer than 100 ms.
• Saccades are fast movements from a point to another (e.g by
skipping a certain number of pixels in the vertical and horizontal
directions on a screen).
Eye tracking: a short history and definitions
8

9. Eye tracking: a short history and definitions
• An example of fixations and saccades
over text.
• This is the typical pattern of eye
movement during reading.
• The eyes never move smoothly over
still text.
http://www.digitimes.com.tw/tw/B2B/Seminar/Service/download/053A410120/053A410120_R559FP5NJ1W8L7F4MCOL.pdf
9

10. Gazeplots and heat-maps
Eye tracking technology has been a useful tool
to evaluate the effectiveness of websites to
communicate information.
In eye tracking research related to HCI it is also
commonly create gazeplots and heat-maps.
https://vwo.com/blog/heatmap-and-ux/
http://www.digitimes.com.tw/tw/B2B/Seminar/Service/download/053A410120
/053A410120_R559FP5NJ1W8L7F4MCOL.pdf

11. Kinds of Eye trackers
Electric potential
based tracking
Eye-attached tracking Optical tracking
http://www.jessebandersen.com/2014/03/the-eye-tribe-development-kit-box.html
11

12. Eye-tracking system
Get a modern PowerPoint Presentation that is beautifully designed. I hope
and I believe that this Template will your Time, Money and Reputation.
Easy to change colors, photos and Text.
Easy to change colors, photos and Text. Get a modern PowerPoint
Presentation that is beautifully designed. I hope and I believe that this
Template will your Time, Money and Reputation. Easy to change colors,
photos and Text.
I hope and I believe that this Template will your Time, Money and
Reputation. Easy to change colors, photos and Text. Easy to change colors,
photos and Text.
Your Text Here
http://www.tobii.com/group/about/this-is-eye-tracking 12

13. Eye movements, definitions, example measures
https://onlinelibrary.wiley.com/doi/pdf/10.1111/medu
13

14. Eye-tracker: possible usages
Assistive interfaces: e.g., web browsing and writing
Ocular control: e.g. keyboard / mouse simulation
Graphic tools: help in drawing activities
Museums: access information directly with
the eyes
RSVP (Rapid Serial Visual Presentation): browsing high
quantities of images
15

15. Eye-tracker: possible usages
01
02
03
04
Classifiers:
they allow to "train" a statistical
instrument so that,
given input data, it can return
answers about something.
Eye tracking has already been
studied for flight safety by
comparing scan paths and fixation
duration to evaluate the progress of
pilot trainees, for estimating pilots’
skills, for analyzing crew’s joint
attention and shared situational
awareness.
Aviation applications
try to figure out if the user is
understanding something, e.g.
– Pupil dimension
– Saccadic movements
E-Learning:
for Authentication /
Validation process
Biometrics:
16

16. ❑ Observe users’ learning behavior in real time by monitoring
characteristics such as:
❑ objects and areas of interest,
❑ time spent on objects,
❑ frequency of visits, and
❑ sequences in which content is consumed.
❑ Research could be focused on analyzing eye-movement
patterns during learning and linking these patterns with
cognitive processes.
Enhancing E-learning system
with Eye Tracking Technologies
17

17. Enhancing E-learning system
with Eye Tracking Technologies
• Fixation duration and saccade length can determine the
learner’s mental state, whether the learner is
concentrating, stressed, tired.
• It can provide encouragement to learners, or even offer
additional material to keep the attention.
• At the same time, the system can let the teacher know
which part of the lesson needs to be improved.
Determining a learner’s mental state.
18

18. ❑ Saccadic velocity - decrease with increasing tiredness and to
increase with increasing task difficulty.
❑ Blink rate - decreasing blink velocity and decreasing degree
of openness may be indicators for increasing tiredness.
❑ Thus, if tiredness is identified, it should be possible through
adaptive e-learning mechanisms to suggest optimized
strategies such as the best time to take a break.
Enhancing E-learning system
with Eye Tracking Technologies
19

19. In an e-learning course concerned with Alexander the Great's Conquest of Persia, a map of
Alexander's advance in the region is shown. The map content is updated in correspondence
to the text paragraph currently read by the learner. In the example, the second paragraph
("Granikos") is being read and the map shows the journey of Alexander from Macedonia to
Granikos (green, yellow and red areas indicate fixations and gaze duration).
Enhancing E-learning system
with Eye Tracking Technologies
20
https://old.eurodl.org

20. Identifying important teaching areas.
• By tracking the number of fixation on each
area of interest, a teacher can make sure
that learners are paying attention to the
most important areas.
• If that is not the case, a teacher can
modify the material.
• Also, if the learner spends too much time
in one area of interest, it could mean that
area is problematic a more detailed
explanation or more related material
should be included.
Enhancing E-learning system
with Eye Tracking Technologies
21

21. ❑ The system can direct the activities of learners and generates referrals
links and actions in the learning process.
❑ System can identify where the gaze is held and can offer a more detailed
explanation of the term or concept in the form of dictionary.
❑ Option to the student himself chooses a word he wants to see details -
button ’Q’.
❑ Identifies and analyses learning sequences and suggests learners
optimal sequence according to the successful results of finally test.
❑ Reports about progress, test results, coursework and their own
learning styles.
Enhancing E-learning system
with Eye Tracking Technologies
22

22. Enhancing E-learning system with Eye
Tracking Technologies
Learning style identification.
• Currently, learners are required to fill in a
questionnaire and we have no way of knowing
if they did it honestly.
• By using eye tracking technologies to identify
learning styles (using a sample lesson), we can
be sure that the learning style they areas
signed with is the one that suits them best.
• Another option would be modifying the style
throughout the course progression.
23

23. ❑ The learning process will be improved, because the system
will create or deliver adapted content by means of tracked
statistical data - optimise material to an individual's needs.
❑ (e.g. by delivering more images/tables for learners that have
problems with large and complicated texts).
❑ if somebody prefers text and ignores pictures the number of
pictures presented could be reduced, and vice versa – learning
style theory – automatic identification of learning styles with
the assistance of eye-tracker
Enhancing E-learning system
with Eye Tracking Technologies
24

24. Enhancing E-learning system with Eye
Tracking Technologies
taking into concern the learner's learning type, perception, cognition, fields of
interest and knowledge level,
supplying personalized course content,
achieving a better quality of the progress throw the course,
providing more details about the perceptive and mental processes of the
learner, recognizing potential learner problems and providing recommendations
for improvement and adaptation, uncovering the need for additional material.
25

25. Assessing Learning Styles Through Eye Tracking for
E-Learning Applications
N. Nugrahaningsih, M. Porta, A. Klasnja-Milicevic, Mirjana Ivanovic

26. Assessing Learning Styles Through Eye Tracking
• Adapting the presentation of learning
material to the specific student’s
characteristics is useful to improve the
overall learning experience and learning
styles can play an important role to this
purpose.
• The study about the possibility to distinguish
between Visual and Verbal learning styles
from gaze data.
26

27. Learning style identification - Felder and Soloman
27

28. Research question
RQ Is it possible to distinguish Visual
and Verbal learners from:
- the features of their gaze behavior
- percentage of fixation duration,
- percentage of fixations, and
- average fixation duration
recorded by an eye tracker?
28

29. Participants
• 90 volunteer students participated in the experiment
• 57 males and 33 females,
• 18 years old on average.
• All of them were freshman Computer Engineering students
of the Informatics Department of the University of
Palangkaraya and had not attended any computer
programming course yet.
• No personal data were stored, as all the participants in the
experiment were anonymously identified through numbers.
• The participants did not get any academic credits for
participating in the experiments.
29

30. Participants
• Six of the 90 participants did not fill in the questionnaire
completely.
• Other four participants failed the eye tracking calibration
procedure (consisting in fixating the center of a circle
appearing in different positions of the screen).
• Moreover, 25 participants tried the test more than once, due
to problems occurring in the data recording phase.
• Thus, in the end, we decided to consider eye data from the
surely reliable 55 participants.
30

31. Materials
To record gaze data - Eye Tribe ET-1000 eye tracker, with 60 Hz data
sampling rate.
Stimuli were displayed on a 21.5'' monitor.
31

32. Procedure
• “Traditional” approach - the participants were initially asked to complete the Index of
Learning Styles (ILS) questionnaire.
Experimental Phase 1.
• After three days from the Experimental Phase 1 - an eye tracking experiment.
• The participants were not informed that this trial related to the questionnaire they
had answered in Phase 1.
• A within-subjects experimental design was used, in which participants tried all the
available conditions.
Experimental Phase 2.
32

33. Experimental Phase 1.
• The ILS questionnaire is an instrument composed of
44 multiple-choice questions which aims to distinguish
four bipolar styles:
• Active/Reflective (AR),
• Sensible/Intuitive (SI),
• Visual/Verbal (VV), and
• Sequential/Global (SG).
• There are two answers (a and b) for each question. In
our study, the original questionnaire was translated into
Indonesian.
33

34. Experimental Phase 1.
The result score is an odd number between 1 and 11:
• If the score is 1 or 3: the respondent is well balanced
on the two dimensions of that scale
• If the score is 5 or 7: the respondent has a moderate
preference for one dimension of the scale
• If the score is 9 or 11: the respondent has a very
strong preference for one dimension of the scale
34

35. Experimental Phase 2.
The eye tracking experiment was conducted in a quiet room,
with artificial illumination from the ceiling.
The participant in the test was seated at about 55 cm from
the monitor.
The task was to read and try to understand the topics
presented in a group of slides.
No time limit was set for each slide, so that the participants
could learn at their own pace (a new slide was loaded by
pressing the space bar). In total there were seven slides.
35

36. Experimental Phase 2.
01
contained a description of the task;
The first slide
02
consisted of a graphical overview of the topics;
The second slide
03
explained the basic notion of variable;
The third slide
04
presented the concept of algorithm;
The fourth slide
covered the three basic imperative programming constructs, namely
sequence, selection, and iteration.
The fifth, sixth and seventh slides, respectively
05
36

37. (a) (b)
(c) (d)
37

38. Analysis of Eye Tracking Data and Results
?
• In each slide, we defined two AOIs: one for the text section and another for the picture region.
• Text and pictures were alternately on the left and on the right within slides.
38

39. Analysis of Eye Tracking Data and Results
• The independent variables of the eye tracking study were the position of the
picture and of the text areas on the slides (left-right or right-left).
• The controlled variables were the textual and graphical contents displayed in
the slides.
• The dependent variables, besides the questionnaire outcomes for Phase 1,
in Phase 2 were:
• the percentage of fixation duration,
• the percentage of fixations, and
• the average fixation duration.
• Percentages were preferred to absolute values because the
time spent on each slide by each participant was different.
39

40. Analysis of Eye Tracking Data and Results
40

41. Analysis of Eye Tracking Data and Results
• For a temporal analysis of eye behavior, we subdivided the whole time spent by
each participant on a slide into ten intervals.
• Score Distributions
• The scores obtained from the Felder-Silverman questionnaire were not evenly
distributed.
• We grouped them based on the median (MED) and median absolute deviation
(MAD) values of the score. Specifically, we identified three groups:
• Group 1, with score < MED – MAD - “more verbal than visual”
• Group 2, with score > MED + MAD - “more visual than verbal”
• Group 3, with score in the range (MED – MAD)  (MED + MAD) –
“between visual and verbal”
42

42. Analysis of Eye Tracking Data and Results
• The answer to the research question of our study (“Is it possible to distinguish Vis
ual and Verbal learners from the features of the gaze behavior recorded by an eye
tracker?”):
• A relation between gaze behavior and learners’ group could be found:
• for Group 1 (participants who were classified as more verbal than visual)
• for Group 2 (participants who were classified as more visual than verbal),
• not for Group 3 (participants who were classified as being between visual and
verbal).
• Specifically, the percentage of fixation duration on the text area, computed up to
intervals 9 and 10 gives clear information about the user’s style group (Group 1 or
Group 2).
• If most of the time (at least 90%) spent on the slide is evaluated, the Visual/Verbal
learner can be successfully recognized.
43

43. Analysis of Eye Tracking Data and
Results
• Gaze data were coupled with the outcomes of the Index of
Learning Styles (ILS) questionnaire.
• A connection between the Visual/Verbal learning style was
found for a specific information layout, which give a
constructive contribution to the field of e-learning in general,
and to the area of automatic learning style assessment
specifically.
• Exploiting eye tracking in this field is of importance because
it can enable “intelligent” e-learning systems in which learning
styles are assessed in a seamless way.
46

44. Conclusions and Future work
The automatic recognition of users’ learning styles is a very important step
towards intelligent adaptive learning platforms.
To achieve an adaptive eLearning system, it is essential to monitor the learner
behavior dynamically to diagnose their learning style.
Eye tracking can serve that purpose by investigating the eye gaze movement
while engaging in the eLearning environment.
It would be also useful to consider an application of eye tracking technology in
combination with other biosensor systems.
Additional tools and analytical data might explore hidden patterns in user behavior
and activities.
47

45. Thank you
Eye-tracking Techniques and Methods in e-Learning Environments