SlideShare a Scribd company logo
1 of 30
Download to read offline
An Empirical Study on the Efficiency of
Graphical vs. Textual Representations in
Requirements Comprehension
Zohreh Sharafi, Alessandro Marchetto,
Angelo Susi, Giuliano Antoniol and YannGaël Guéhéneuc

ICPC’13
Outline
Introduction
Problem and Motivation
Related Work
Our Empirical Study
Design
Result and Analysis
Threats to Validity

Conclusion
2
Introduction
Documentation is often only
textual or graphical

Graphical
Documentation

Comments
Manual pages

Not always available
Requirements

3

Program comprehension relies
heavily on documentation
Introduction
Is there an impact (accuracy, time) of the kind
of document representation on program
comprehension?
Requirements
Structured text only
Graphical representation only
Mixed textual and graphical representation

Is there a impact of the mother language,
degree of study, or gender?
4
Requirement Representation
Requirement
Textual

Graphical (TROPOS)

Accuracy

Higher accuracy or
no difference?

Time

More time spent on
textual or more on graphical?

Effort

Visual effort ?

5
Gender Preference
Does gender affects the preferred document
representation?
Does gender and representation affect
program comprehension?

6
Related Works
Ottensooser et al. [3]

Textual representations

Significant improvement in
understanding of business processes
when subjects work with textual
representations

Somervell et al. [12]

Combination of graphical and textual
representations were more efficient

Guidelines on the use of a
combination of textual and graphical
representations to improve subjects’
efficiency

Razali et al. [13]

Graphical formal specification vs. a
purely textual formal specification

Combination of semi-formal and
formal notations improves the
subjects’ accuracy

Heijstek [2] et al.

Graphical and textual notations for
software architecture

No difference in accuracy, more
experienced subjects mostly
preferred a textual representation

[2]

[3]
[12]
[13]

7

W. Heijstek, T. Kuhne, and M. R. V. Chaudron, “Experimental analysis of textual and graphical representations for software architecture
design,” in Proceedings of the International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, 2011, pp.
167–176.
A. Ottensooser, A. Fekete, H. A. Reijers, J. Mendling, and C. Menictas, “Making sense of business process descriptions: An experimental
comparison of graphical and textual notations,” Journal of Systems and Software, vol. 85, no. 3, pp. 596–606, March 2012.
J. Somervell, C. M. Chewar, and D. S. Mccrickard, “Evaluating graphical vs. textual secondary displays for information notification,” in
Proceedings of the ACM Southeast Conference, ACM press, 2002, pp. 153–160
C. F. Snook and R. Harrison, “Experimental comparison of the comprehensibility of a UML-based formal specification versus a textual one,” in
Proceedings of 11th International Conference on Evaluation and Assessment in Software Engineering, ACM Press, 2007, pp. 955–971.
TROPOS
TROPOS is a goal-oriented requirements modeling
approach based on concepts such as:
Actor - typically representing a domain stakeholder
Goal - representing a state of affairs desired by the actor
Task- representing set of activities which operationalizes goals
Resource - which is an element (such as information, device, database, …)
whose presence is needed to support the satisfaction of goals or the
execution of a task

And relationships such as:
AND/OR decomposition of goals and tasks into sub-goals and sub-tasks
Means-ends to describe the relationship between a goal and the task that
fulfill it

Each concept or relationship has a visual counterpart
8
TROPOS: goal diagrams
And
decomposition
of goals

Actor

Goal

Task

Resource
Means-ends
relationship
9
Our Empirical Study
Our Goal: Design and perform an experiment to investigate
the impact of requirement representation on comprehension
accuracy, time and strategy.

High Level Research Question: Does the document
representation impact
understanding tasks ?

Perspective:
Developers
Researcher
10

time

or

accuracy

in

program
Detailed Research Questions
RQ1: Does the type of requirement representations
(graphical vs. textual) impact the developers' effort,
time, and answer accuracy in requirements
comprehension tasks?
RQ2: Does the structure of the representations lead
developers to use specific task-solving strategies
(top-down vs. bottom-up) during requirements
comprehension tasks?
RQ3: Given a graphical and textual representation of
a requirements comprehension task, is there any
preferred representation by the subjects?
11
Detailed Research Questions
RQ2: Does the structure of the
representations lead developers to use specific
task-solving strategies (top-down vs. bottomup) during requirements comprehension tasks?

12
Detailed Research Questions
RQ3: Given a graphical and textual
representation of a requirements
comprehension task, is there any preferred
representation by the subjects?

13
Experiment Design
Goal

Study the impact of requirement representation

Independent 1. Document representation
variables
a) Graphical, b) Textual, c) Both;
Dependent
variables

1. Accuracy
2. Required time
3. Effort - Visual Effort

Mitigating
variables

1.
2.
3.
4.

Study level
English language proficiency
Mother language
Gender: male (M) or female (F)
Subjects’ Demography

Academic background

Gender

Ph.D.

B.Sc.

Male

Female

15
14

M.Sc.

11

2

16

12
Experiment Design
Subjects’ Demography
Academic background

Gender

Ph.D.

B.Sc.

Male

Female

15

15

M.Sc.

11

2

16

12
Experiment Design

16
Experiment Design

17
Experiment Design
FaceLAB
Video-based
Two camera
One infrared

Non-intrusive
No goggles
No wires
No sensing device
18
Result and Analysis: Visual Effort
Visual effort
Calculated from eye-tracking data.
Calculated based on the amount of visual attention
less attention
less time
less effort

Visual attention triggers the mental processes
Two types of eye gaze data
Fixation
Saccade

We use fixation to calculate effort
19
Result and Analysis: Visual Effort
Convex hull: the smallest convex sets of fixations
that contains all of a subject’s fixations*
Measure Average Fixation Duration (AFD) via convex
hull as effort proxy
Smaller convex hull
close fixations
less effort

20

*

J. H. Goldberg and X. P. Kotval, “Computer interface evaluation using eye movements”, 1999.
Result and Analysis: Visual Effort

21
Result and Analysis: RQ1
Accuracy %
Correct

Wrong

Graphical

97%

3%

Textual

98%

2%

Mixed

96%

4%

No significant difference
Accuracy

Time
22

There is a significant
difference
Result and Analysis: RQ1
Different model imply different areas of focus
Result and Analysis: RQ1
There is a significant difference in visual effort though
Cohen-d is from medium up
AFD(Q) -- Average Fixation Duration -- is borderline 0.07 !

M:
Re:
Ir:
Q:
24

Model
Relevant
Irrelevant
Question
Result and Analysis: RQ2
5
1
Goal area

2
Task area

3
Resource
area

4

25
Result and Analysis: RQ2
The structure of our document makes subject use a top-down
(goals to resources) or bottom up (resources to goals)
strategy

26
Result and Analysis: RQ3
Subject Prefer Graphic Notation

27
Threats to validity
Internal validity
Random ordering of stimuli
Provide comfortable environment

External validity (generalisation of the results)
Students as subjects
“Only” 28 subjects

28
Conclusion
Requirement representation has an impact
Accuracy

Time

Effort

Preference

Language distance has no impact
Closer to English is “better”

Accuracy

Time

Graphical representation is preferred but
requires greater effort
Gender has not impact
29
Conclusion
Design and perform an eye-tracking
experiment
Investigate the impact of document
representation on program comprehension
Examine accuracy, time, effort, and preference

30

More Related Content

What's hot

Empirical study of programming to an interface
Empirical study of programming to an interfaceEmpirical study of programming to an interface
Empirical study of programming to an interfacefuhrmanator
 
Deep learning based code smell detection - Qualifying Talk
Deep learning based code smell detection - Qualifying TalkDeep learning based code smell detection - Qualifying Talk
Deep learning based code smell detection - Qualifying TalkSayed Mohsin Reza
 
A Metric for Code Readability
A Metric for Code ReadabilityA Metric for Code Readability
A Metric for Code ReadabilityRay Buse
 
DETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALE
DETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALEDETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALE
DETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALEijseajournal
 
The Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for HindiThe Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for Hindisinghg77
 
NLP Project Presentation
NLP Project PresentationNLP Project Presentation
NLP Project PresentationAryak Sengupta
 
Dupressoir
DupressoirDupressoir
Dupressoiranesah
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingYoung Seok Kim
 
Seq2seq Model to Tokenize the Chinese Language
Seq2seq Model to Tokenize the Chinese LanguageSeq2seq Model to Tokenize the Chinese Language
Seq2seq Model to Tokenize the Chinese LanguageJinho Choi
 
08 choosing connectors
08 choosing connectors08 choosing connectors
08 choosing connectorsMajong DevJfu
 
Deep Learning | Speaker Indentification
Deep Learning | Speaker IndentificationDeep Learning | Speaker Indentification
Deep Learning | Speaker IndentificationSai Kiran Kadam
 
The I in PRIMM - Code Comprehension and Questioning
The I in PRIMM - Code Comprehension and QuestioningThe I in PRIMM - Code Comprehension and Questioning
The I in PRIMM - Code Comprehension and QuestioningSue Sentance
 
Deep Learning - Speaker Recognition
Deep Learning - Speaker Recognition Deep Learning - Speaker Recognition
Deep Learning - Speaker Recognition Sai Kiran Kadam
 
Summarization Techniques for Code, Changes, and Testing
Summarization Techniques for Code, Changes, and TestingSummarization Techniques for Code, Changes, and Testing
Summarization Techniques for Code, Changes, and TestingSebastiano Panichella
 
The effect of number of concepts on readability of schemas 2
The effect of number of concepts on readability of schemas 2The effect of number of concepts on readability of schemas 2
The effect of number of concepts on readability of schemas 2Saman Sara
 
GPT-2: Language Models are Unsupervised Multitask Learners
GPT-2: Language Models are Unsupervised Multitask LearnersGPT-2: Language Models are Unsupervised Multitask Learners
GPT-2: Language Models are Unsupervised Multitask LearnersYoung Seok Kim
 

What's hot (20)

Icsm19.ppt
Icsm19.pptIcsm19.ppt
Icsm19.ppt
 
Empirical study of programming to an interface
Empirical study of programming to an interfaceEmpirical study of programming to an interface
Empirical study of programming to an interface
 
Icpc11c.ppt
Icpc11c.pptIcpc11c.ppt
Icpc11c.ppt
 
Deep learning based code smell detection - Qualifying Talk
Deep learning based code smell detection - Qualifying TalkDeep learning based code smell detection - Qualifying Talk
Deep learning based code smell detection - Qualifying Talk
 
Icsm20.ppt
Icsm20.pptIcsm20.ppt
Icsm20.ppt
 
DeepPavlov 2019
DeepPavlov 2019DeepPavlov 2019
DeepPavlov 2019
 
A Metric for Code Readability
A Metric for Code ReadabilityA Metric for Code Readability
A Metric for Code Readability
 
DETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALE
DETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALEDETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALE
DETECTION AND REFACTORING OF BAD SMELL CAUSED BY LARGE SCALE
 
The Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for HindiThe Essay Scoring Tool (TEST) for Hindi
The Essay Scoring Tool (TEST) for Hindi
 
NLP Project Presentation
NLP Project PresentationNLP Project Presentation
NLP Project Presentation
 
Dupressoir
DupressoirDupressoir
Dupressoir
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
 
Seq2seq Model to Tokenize the Chinese Language
Seq2seq Model to Tokenize the Chinese LanguageSeq2seq Model to Tokenize the Chinese Language
Seq2seq Model to Tokenize the Chinese Language
 
08 choosing connectors
08 choosing connectors08 choosing connectors
08 choosing connectors
 
Deep Learning | Speaker Indentification
Deep Learning | Speaker IndentificationDeep Learning | Speaker Indentification
Deep Learning | Speaker Indentification
 
The I in PRIMM - Code Comprehension and Questioning
The I in PRIMM - Code Comprehension and QuestioningThe I in PRIMM - Code Comprehension and Questioning
The I in PRIMM - Code Comprehension and Questioning
 
Deep Learning - Speaker Recognition
Deep Learning - Speaker Recognition Deep Learning - Speaker Recognition
Deep Learning - Speaker Recognition
 
Summarization Techniques for Code, Changes, and Testing
Summarization Techniques for Code, Changes, and TestingSummarization Techniques for Code, Changes, and Testing
Summarization Techniques for Code, Changes, and Testing
 
The effect of number of concepts on readability of schemas 2
The effect of number of concepts on readability of schemas 2The effect of number of concepts on readability of schemas 2
The effect of number of concepts on readability of schemas 2
 
GPT-2: Language Models are Unsupervised Multitask Learners
GPT-2: Language Models are Unsupervised Multitask LearnersGPT-2: Language Models are Unsupervised Multitask Learners
GPT-2: Language Models are Unsupervised Multitask Learners
 

Viewers also liked

Ase07 tooldemo.ppt
Ase07 tooldemo.pptAse07 tooldemo.ppt
Ase07 tooldemo.pptPtidej Team
 
130918 maelick claes - ecological studies of open source software ecosystems
130918   maelick claes - ecological studies of open source software ecosystems130918   maelick claes - ecological studies of open source software ecosystems
130918 maelick claes - ecological studies of open source software ecosystemsPtidej Team
 
130823 eric vitouley - supporting source code maintenance using screencasts
130823   eric vitouley - supporting source code maintenance using screencasts130823   eric vitouley - supporting source code maintenance using screencasts
130823 eric vitouley - supporting source code maintenance using screencastsPtidej Team
 
Cascon06 tooldemo.ppt
Cascon06 tooldemo.pptCascon06 tooldemo.ppt
Cascon06 tooldemo.pptPtidej Team
 
130924 yann-gael gueheneuc - an overview of software code quality and conne...
130924   yann-gael gueheneuc - an overview of software code quality and conne...130924   yann-gael gueheneuc - an overview of software code quality and conne...
130924 yann-gael gueheneuc - an overview of software code quality and conne...Ptidej Team
 

Viewers also liked (7)

Wcre12a.ppt
Wcre12a.pptWcre12a.ppt
Wcre12a.ppt
 
Ase07 tooldemo.ppt
Ase07 tooldemo.pptAse07 tooldemo.ppt
Ase07 tooldemo.ppt
 
130918 maelick claes - ecological studies of open source software ecosystems
130918   maelick claes - ecological studies of open source software ecosystems130918   maelick claes - ecological studies of open source software ecosystems
130918 maelick claes - ecological studies of open source software ecosystems
 
130823 eric vitouley - supporting source code maintenance using screencasts
130823   eric vitouley - supporting source code maintenance using screencasts130823   eric vitouley - supporting source code maintenance using screencasts
130823 eric vitouley - supporting source code maintenance using screencasts
 
Cascon06 tooldemo.ppt
Cascon06 tooldemo.pptCascon06 tooldemo.ppt
Cascon06 tooldemo.ppt
 
Ppap13b.ppt
Ppap13b.pptPpap13b.ppt
Ppap13b.ppt
 
130924 yann-gael gueheneuc - an overview of software code quality and conne...
130924   yann-gael gueheneuc - an overview of software code quality and conne...130924   yann-gael gueheneuc - an overview of software code quality and conne...
130924 yann-gael gueheneuc - an overview of software code quality and conne...
 

Similar to Icpc13.ppt

Learning about Information Searchers from Eye-Tracking by Jacek Gwizdka
Learning about Information Searchers from Eye-Tracking by Jacek GwizdkaLearning about Information Searchers from Eye-Tracking by Jacek Gwizdka
Learning about Information Searchers from Eye-Tracking by Jacek Gwizdkajacekg
 
Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...haochuan
 
Quantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf Weighting Quantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf Weighting ijistjournal
 
Quantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf WeightingQuantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf Weightingijistjournal
 
Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)
Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)
Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)Michael Derntl
 
Sign Language Recognition based on Hands symbols Classification
Sign Language Recognition based on Hands symbols ClassificationSign Language Recognition based on Hands symbols Classification
Sign Language Recognition based on Hands symbols ClassificationTriloki Gupta
 
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docx
Due Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docxDue Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docx
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docxgertrudebellgrove
 
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docx
 Due Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docx Due Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docx
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docxShiraPrater50
 
COMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUE
COMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUECOMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUE
COMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUEJournal For Research
 
The Prepared Executive: A Linguistic Exploration
The Prepared Executive: A Linguistic ExplorationThe Prepared Executive: A Linguistic Exploration
The Prepared Executive: A Linguistic ExplorationTom Donoghue
 
Effects of Individual Success on Globally Distributed Team Performance
Effects of Individual Success on Globally Distributed Team PerformanceEffects of Individual Success on Globally Distributed Team Performance
Effects of Individual Success on Globally Distributed Team PerformanceOnur Yılmaz
 
Feldt Ist Revised 090920 Links Betwe
Feldt Ist Revised 090920 Links BetweFeldt Ist Revised 090920 Links Betwe
Feldt Ist Revised 090920 Links Betwericardocruzjr
 
Analysis of the learning object-oriented programming factors
Analysis of the learning object-oriented programming factors Analysis of the learning object-oriented programming factors
Analysis of the learning object-oriented programming factors IJECEIAES
 
Diagrammatic knowledge modeling for managers – ontology-based approach
Diagrammatic knowledge modeling for managers  – ontology-based approachDiagrammatic knowledge modeling for managers  – ontology-based approach
Diagrammatic knowledge modeling for managers – ontology-based approachDmitry Kudryavtsev
 
A Preliminary Study Of Applying ERP On Users Reactions To Web Pages With Dif...
A Preliminary Study Of Applying ERP On Users  Reactions To Web Pages With Dif...A Preliminary Study Of Applying ERP On Users  Reactions To Web Pages With Dif...
A Preliminary Study Of Applying ERP On Users Reactions To Web Pages With Dif...Scott Bou
 
Live Sign Language Translation: A Survey
Live Sign Language Translation: A SurveyLive Sign Language Translation: A Survey
Live Sign Language Translation: A SurveyIRJET Journal
 
Quality Of The Software Engineering
Quality Of The Software EngineeringQuality Of The Software Engineering
Quality Of The Software EngineeringPatty Buckley
 
Games to Improve Clinical Practice and Healthcare Administration
Games to Improve Clinical Practice and Healthcare AdministrationGames to Improve Clinical Practice and Healthcare Administration
Games to Improve Clinical Practice and Healthcare AdministrationSeriousGamesAssoc
 

Similar to Icpc13.ppt (20)

Icpc13.ppt
Icpc13.pptIcpc13.ppt
Icpc13.ppt
 
Learning about Information Searchers from Eye-Tracking by Jacek Gwizdka
Learning about Information Searchers from Eye-Tracking by Jacek GwizdkaLearning about Information Searchers from Eye-Tracking by Jacek Gwizdka
Learning about Information Searchers from Eye-Tracking by Jacek Gwizdka
 
Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...
 
Quantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf Weighting Quantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf Weighting
 
Quantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf WeightingQuantification of Portrayal Concepts using tf-idf Weighting
Quantification of Portrayal Concepts using tf-idf Weighting
 
FLOSS Pilot Studies
FLOSS Pilot StudiesFLOSS Pilot Studies
FLOSS Pilot Studies
 
Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)
Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)
Basics of Paper Writing and Publishing in TEL (JTEL 2015 Workshop)
 
Sign Language Recognition based on Hands symbols Classification
Sign Language Recognition based on Hands symbols ClassificationSign Language Recognition based on Hands symbols Classification
Sign Language Recognition based on Hands symbols Classification
 
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docx
Due Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docxDue Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docx
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docx
 
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docx
 Due Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docx Due Date  1159 p.m. EST, Sunday of Unit 7 Points  10.docx
Due Date 1159 p.m. EST, Sunday of Unit 7 Points 10.docx
 
COMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUE
COMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUECOMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUE
COMPREHENSIVE ANALYSIS OF NATURAL LANGUAGE PROCESSING TECHNIQUE
 
The Prepared Executive: A Linguistic Exploration
The Prepared Executive: A Linguistic ExplorationThe Prepared Executive: A Linguistic Exploration
The Prepared Executive: A Linguistic Exploration
 
Effects of Individual Success on Globally Distributed Team Performance
Effects of Individual Success on Globally Distributed Team PerformanceEffects of Individual Success on Globally Distributed Team Performance
Effects of Individual Success on Globally Distributed Team Performance
 
Feldt Ist Revised 090920 Links Betwe
Feldt Ist Revised 090920 Links BetweFeldt Ist Revised 090920 Links Betwe
Feldt Ist Revised 090920 Links Betwe
 
Analysis of the learning object-oriented programming factors
Analysis of the learning object-oriented programming factors Analysis of the learning object-oriented programming factors
Analysis of the learning object-oriented programming factors
 
Diagrammatic knowledge modeling for managers – ontology-based approach
Diagrammatic knowledge modeling for managers  – ontology-based approachDiagrammatic knowledge modeling for managers  – ontology-based approach
Diagrammatic knowledge modeling for managers – ontology-based approach
 
A Preliminary Study Of Applying ERP On Users Reactions To Web Pages With Dif...
A Preliminary Study Of Applying ERP On Users  Reactions To Web Pages With Dif...A Preliminary Study Of Applying ERP On Users  Reactions To Web Pages With Dif...
A Preliminary Study Of Applying ERP On Users Reactions To Web Pages With Dif...
 
Live Sign Language Translation: A Survey
Live Sign Language Translation: A SurveyLive Sign Language Translation: A Survey
Live Sign Language Translation: A Survey
 
Quality Of The Software Engineering
Quality Of The Software EngineeringQuality Of The Software Engineering
Quality Of The Software Engineering
 
Games to Improve Clinical Practice and Healthcare Administration
Games to Improve Clinical Practice and Healthcare AdministrationGames to Improve Clinical Practice and Healthcare Administration
Games to Improve Clinical Practice and Healthcare Administration
 

More from Ptidej Team

From IoT to Software Miniaturisation
From IoT to Software MiniaturisationFrom IoT to Software Miniaturisation
From IoT to Software MiniaturisationPtidej Team
 
Presentation by Lionel Briand
Presentation by Lionel BriandPresentation by Lionel Briand
Presentation by Lionel BriandPtidej Team
 
Manel Abdellatif
Manel AbdellatifManel Abdellatif
Manel AbdellatifPtidej Team
 
Azadeh Kermansaravi
Azadeh KermansaraviAzadeh Kermansaravi
Azadeh KermansaraviPtidej Team
 
CSED - Manel Grichi
CSED - Manel GrichiCSED - Manel Grichi
CSED - Manel GrichiPtidej Team
 
Cristiano Politowski
Cristiano PolitowskiCristiano Politowski
Cristiano PolitowskiPtidej Team
 
Will io t trigger the next software crisis
Will io t trigger the next software crisisWill io t trigger the next software crisis
Will io t trigger the next software crisisPtidej Team
 
Thesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.pptThesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.pptPtidej Team
 
Thesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.pptThesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.pptPtidej Team
 

More from Ptidej Team (20)

From IoT to Software Miniaturisation
From IoT to Software MiniaturisationFrom IoT to Software Miniaturisation
From IoT to Software Miniaturisation
 
Presentation
PresentationPresentation
Presentation
 
Presentation
PresentationPresentation
Presentation
 
Presentation
PresentationPresentation
Presentation
 
Presentation by Lionel Briand
Presentation by Lionel BriandPresentation by Lionel Briand
Presentation by Lionel Briand
 
Manel Abdellatif
Manel AbdellatifManel Abdellatif
Manel Abdellatif
 
Azadeh Kermansaravi
Azadeh KermansaraviAzadeh Kermansaravi
Azadeh Kermansaravi
 
Mouna Abidi
Mouna AbidiMouna Abidi
Mouna Abidi
 
CSED - Manel Grichi
CSED - Manel GrichiCSED - Manel Grichi
CSED - Manel Grichi
 
Cristiano Politowski
Cristiano PolitowskiCristiano Politowski
Cristiano Politowski
 
Will io t trigger the next software crisis
Will io t trigger the next software crisisWill io t trigger the next software crisis
Will io t trigger the next software crisis
 
MIPA
MIPAMIPA
MIPA
 
Thesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.pptThesis+of+laleh+eshkevari.ppt
Thesis+of+laleh+eshkevari.ppt
 
Thesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.pptThesis+of+nesrine+abdelkafi.ppt
Thesis+of+nesrine+abdelkafi.ppt
 
Medicine15.ppt
Medicine15.pptMedicine15.ppt
Medicine15.ppt
 
Qrs17b.ppt
Qrs17b.pptQrs17b.ppt
Qrs17b.ppt
 
Icpc11c.ppt
Icpc11c.pptIcpc11c.ppt
Icpc11c.ppt
 
Icsme16.ppt
Icsme16.pptIcsme16.ppt
Icsme16.ppt
 
Msr17a.ppt
Msr17a.pptMsr17a.ppt
Msr17a.ppt
 
Icsoc15.ppt
Icsoc15.pptIcsoc15.ppt
Icsoc15.ppt
 

Recently uploaded

Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 

Recently uploaded (20)

Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 

Icpc13.ppt

  • 1. An Empirical Study on the Efficiency of Graphical vs. Textual Representations in Requirements Comprehension Zohreh Sharafi, Alessandro Marchetto, Angelo Susi, Giuliano Antoniol and YannGaël Guéhéneuc ICPC’13
  • 2. Outline Introduction Problem and Motivation Related Work Our Empirical Study Design Result and Analysis Threats to Validity Conclusion 2
  • 3. Introduction Documentation is often only textual or graphical Graphical Documentation Comments Manual pages Not always available Requirements 3 Program comprehension relies heavily on documentation
  • 4. Introduction Is there an impact (accuracy, time) of the kind of document representation on program comprehension? Requirements Structured text only Graphical representation only Mixed textual and graphical representation Is there a impact of the mother language, degree of study, or gender? 4
  • 5. Requirement Representation Requirement Textual Graphical (TROPOS) Accuracy Higher accuracy or no difference? Time More time spent on textual or more on graphical? Effort Visual effort ? 5
  • 6. Gender Preference Does gender affects the preferred document representation? Does gender and representation affect program comprehension? 6
  • 7. Related Works Ottensooser et al. [3] Textual representations Significant improvement in understanding of business processes when subjects work with textual representations Somervell et al. [12] Combination of graphical and textual representations were more efficient Guidelines on the use of a combination of textual and graphical representations to improve subjects’ efficiency Razali et al. [13] Graphical formal specification vs. a purely textual formal specification Combination of semi-formal and formal notations improves the subjects’ accuracy Heijstek [2] et al. Graphical and textual notations for software architecture No difference in accuracy, more experienced subjects mostly preferred a textual representation [2] [3] [12] [13] 7 W. Heijstek, T. Kuhne, and M. R. V. Chaudron, “Experimental analysis of textual and graphical representations for software architecture design,” in Proceedings of the International Symposium on Empirical Software Engineering and Measurement, IEEE Computer Society, 2011, pp. 167–176. A. Ottensooser, A. Fekete, H. A. Reijers, J. Mendling, and C. Menictas, “Making sense of business process descriptions: An experimental comparison of graphical and textual notations,” Journal of Systems and Software, vol. 85, no. 3, pp. 596–606, March 2012. J. Somervell, C. M. Chewar, and D. S. Mccrickard, “Evaluating graphical vs. textual secondary displays for information notification,” in Proceedings of the ACM Southeast Conference, ACM press, 2002, pp. 153–160 C. F. Snook and R. Harrison, “Experimental comparison of the comprehensibility of a UML-based formal specification versus a textual one,” in Proceedings of 11th International Conference on Evaluation and Assessment in Software Engineering, ACM Press, 2007, pp. 955–971.
  • 8. TROPOS TROPOS is a goal-oriented requirements modeling approach based on concepts such as: Actor - typically representing a domain stakeholder Goal - representing a state of affairs desired by the actor Task- representing set of activities which operationalizes goals Resource - which is an element (such as information, device, database, …) whose presence is needed to support the satisfaction of goals or the execution of a task And relationships such as: AND/OR decomposition of goals and tasks into sub-goals and sub-tasks Means-ends to describe the relationship between a goal and the task that fulfill it Each concept or relationship has a visual counterpart 8
  • 9. TROPOS: goal diagrams And decomposition of goals Actor Goal Task Resource Means-ends relationship 9
  • 10. Our Empirical Study Our Goal: Design and perform an experiment to investigate the impact of requirement representation on comprehension accuracy, time and strategy. High Level Research Question: Does the document representation impact understanding tasks ? Perspective: Developers Researcher 10 time or accuracy in program
  • 11. Detailed Research Questions RQ1: Does the type of requirement representations (graphical vs. textual) impact the developers' effort, time, and answer accuracy in requirements comprehension tasks? RQ2: Does the structure of the representations lead developers to use specific task-solving strategies (top-down vs. bottom-up) during requirements comprehension tasks? RQ3: Given a graphical and textual representation of a requirements comprehension task, is there any preferred representation by the subjects? 11
  • 12. Detailed Research Questions RQ2: Does the structure of the representations lead developers to use specific task-solving strategies (top-down vs. bottomup) during requirements comprehension tasks? 12
  • 13. Detailed Research Questions RQ3: Given a graphical and textual representation of a requirements comprehension task, is there any preferred representation by the subjects? 13
  • 14. Experiment Design Goal Study the impact of requirement representation Independent 1. Document representation variables a) Graphical, b) Textual, c) Both; Dependent variables 1. Accuracy 2. Required time 3. Effort - Visual Effort Mitigating variables 1. 2. 3. 4. Study level English language proficiency Mother language Gender: male (M) or female (F) Subjects’ Demography Academic background Gender Ph.D. B.Sc. Male Female 15 14 M.Sc. 11 2 16 12
  • 15. Experiment Design Subjects’ Demography Academic background Gender Ph.D. B.Sc. Male Female 15 15 M.Sc. 11 2 16 12
  • 18. Experiment Design FaceLAB Video-based Two camera One infrared Non-intrusive No goggles No wires No sensing device 18
  • 19. Result and Analysis: Visual Effort Visual effort Calculated from eye-tracking data. Calculated based on the amount of visual attention less attention less time less effort Visual attention triggers the mental processes Two types of eye gaze data Fixation Saccade We use fixation to calculate effort 19
  • 20. Result and Analysis: Visual Effort Convex hull: the smallest convex sets of fixations that contains all of a subject’s fixations* Measure Average Fixation Duration (AFD) via convex hull as effort proxy Smaller convex hull close fixations less effort 20 * J. H. Goldberg and X. P. Kotval, “Computer interface evaluation using eye movements”, 1999.
  • 21. Result and Analysis: Visual Effort 21
  • 22. Result and Analysis: RQ1 Accuracy % Correct Wrong Graphical 97% 3% Textual 98% 2% Mixed 96% 4% No significant difference Accuracy Time 22 There is a significant difference
  • 23. Result and Analysis: RQ1 Different model imply different areas of focus
  • 24. Result and Analysis: RQ1 There is a significant difference in visual effort though Cohen-d is from medium up AFD(Q) -- Average Fixation Duration -- is borderline 0.07 ! M: Re: Ir: Q: 24 Model Relevant Irrelevant Question
  • 25. Result and Analysis: RQ2 5 1 Goal area 2 Task area 3 Resource area 4 25
  • 26. Result and Analysis: RQ2 The structure of our document makes subject use a top-down (goals to resources) or bottom up (resources to goals) strategy 26
  • 27. Result and Analysis: RQ3 Subject Prefer Graphic Notation 27
  • 28. Threats to validity Internal validity Random ordering of stimuli Provide comfortable environment External validity (generalisation of the results) Students as subjects “Only” 28 subjects 28
  • 29. Conclusion Requirement representation has an impact Accuracy Time Effort Preference Language distance has no impact Closer to English is “better” Accuracy Time Graphical representation is preferred but requires greater effort Gender has not impact 29
  • 30. Conclusion Design and perform an eye-tracking experiment Investigate the impact of document representation on program comprehension Examine accuracy, time, effort, and preference 30