Presentation at NeuroIS Retreat 2018. Authors : Boutin, Léger, Davis, Hevner, Labonté-LeMoyne.

1. Attentional Characteristics
of Anomaly Detection
in Conceptual Modeling
Karl-David Boutin
Pierre-Majorique Léger, Ph.D.
Élise Labonté-LeMoyne, Ph.D.
HEC Montréal
Christopher J. Davis, Ph.D.
Alan H. Hevner, Ph.D.
University of South Florida
NeuroIS Retreat 2018, Vienna (Austria), June 19th - 21st

2. © Copyright Léger et al. 2018
Conceptual
Modeling
▪▪ Facilitate communication about business
domains and their processes
▪▪ Primary medium used in design activities
▪▪ Support the investigation of problems
and improvement of processes

3. © Copyright Léger et al. 2018
Research goals
Explore the differences in the attentional
characteristics between successful and
unsuccessful diagnostics in a detection task.

4. © Copyright Léger et al. 2018
Literature review
BUSINESS PROCESS MODELING
NOTATION (BPMN)
▪▪ Easily learned for simple use
▪▪ Composed of visual syntax and visual
semantics
▪▪ Compare both the semantic and
syntactic error identification process

5. © Copyright Léger et al. 2018
Literature review
VISUAL ATTENTION
Paradigms Findings Studies
Error detection tasks
Errors arefixated more often
and longer than irrelevant
information
Van Waes & al.(2009) ;
Henderson and
Hollingworth (1999) ;
Holmqvist& al.(2011)
High number of fixations on the
stimulus is correlated with an
ineffective search
Holmqvist& al.(2011) ;
Goldberg and Kotval
(1999)
Expert-novice differences
Experts spend less timelooking
at stimuli beforefixating
relevant areas or anomalies
Gegenfurtner & al.(2011)
; Krupinski (2000) ;
Reingold and Sheridan
(2011) ; Sheridan and
Reingold (2014)
Experts also tend to have fewer
fixations and fixation durations
Gegenfurtner & al.(2011)
; Yusuf & al.(2007) ;
Sheridan and Reingold
(2014)

6. © Copyright Léger et al. 2018
Hypotheses
H1
Successful error detections in
conceptual modeling will require less
time spent looking at the stimulus
than unsuccessful error detections.
H2
Successful error detections in
conceptual modeling will require, in
total, fewer fixations than unsuccessful
error detections, but with a higher
proportion of fixations on the error.
H3
Successful error detections in
conceptual modeling will require, on
average, shorter fixation duration than
unsuccessful error detections, but with
longer fixation duration on the error.

7. © Copyright Léger et al. 2018
Research
method
Fast food
Banking
Air travel
Medical appointment
Online shopping
Stimuli
No error
Semantic error
Syntactic error

8. © Copyright Léger et al. 2018
Stimuli

9. © Copyright Léger et al. 2018
Experimental design

10. © Copyright Léger et al. 2018
Instruments
and Measures
MEASURES
▪▪ Number and duration of fixations
▪▪ Time before the first fixation in an
AOI
▪▪ Total view time
INSTRUMENTS
SMI RED250 Eye Tracking Device

11. © Copyright Léger et al. 2018
Participants
▪▪ 18 participants (7 males, 11 females)
▪▪ Aged between 21 and 53 years
(average 32.05 ; std. Dev. 10.24)
▪▪ 7 students and 11 professionals from
11 different organizations
▪▪ Compensation of 20$ at the university’s
bookstore

12. © Copyright Léger et al. 2018
Results
Significant links between successfully
detecting an error and:
▪▪ Smaller amount of time spent on
a stimulus (H1)
▪▪ Fewer fixations and a greater proportion
of fixations in the areas of interest (H2)
▪▪ Shorter fixation duration (H3)
Semantic AOIs
Measure Estimate (β) P-Value
Fixationcount (H2) -0.04956 0.0897*
Fixationduration (H3) 0.1654 0.0061**
Fixationproportion(H2) 0.5448 < 0.0001****
Time to first fixation -0.3333 0.0027**
*=P<0.1 ; **= P<0.01 ; ***=P<0.001 ; ****= P<0.0001
Syntactic AOIs
Measure Estimate (β) P-Value
Fixationcount (H2) 0.3654 < 0.0001****
Fixationduration (H3) 0.4436 < 0.0001****
Fixationproportion(H2) 0.9379 < 0.0001****
*=P<0.1 ; **= P<0.01 ; ***=P<0.001 ; ****= P<0.0001
All models
Measure Estimate (β) P-Value
Fixationcount (H2) -0.4402 < 0.0001****
Fixationduration (H3) -0.373 < 0.0001****
Total view time (H1) -0.3934 < 0.0001****
*=P<0.1 ; **= P<0.01 ; ***=P<0.001 ; ****= P<0.0001
Effect on visual attention for correct answers

13. Contribution
▪▪ Evolution of BPMN and other notations
▪▪ Recommendations for curriculum
development and training methods.
▪▪ Comparaison of semantic and syntactic
errors

14. © Copyright Léger et al. 2018
Next step
▪▪ Replicate the study in order to have a
bigger sample
▪▪ Study the effect of expertise
▪▪ Change the complexity of the stimuli
▪▪ EEG and other apparatus and measures

15. Thank you!
QUESTIONS?