SlideShare uma empresa Scribd logo
1 de 46
Baixar para ler offline
Methodology and Tools for Research:
Reading scientific material
Yannick Prié
Polytech Nantes, University of Nantes
Master DMKM, 2015-2016
CC	BY-SA	4.0
Attribution-ShareAlike 4.0
International (CC BY-SA 4.0)
•  This course "Methodology and Tools for Research:
Reading Scientific Material" by Yannick Prié is licensed
CC BY-SA 4.0
•  This license covers the general organization of the
material, the textual content, the figures, etc. except
where indicated.
•  This license means that you can share and adapt this
course, provided you give appropriate credit to the
author and distribute your contributions under the same
license as the original
◦  for more information about this license, see
http://creativecommons.org/licenses/by-sa/4.0/
Objectives of this course
•  How to read a paper?
•  Various kinds of validations
•  Kinds of writings related to reading
◦  Active reading and annotations
◦  Reading notes
◦  Synthesis
◦  Reviews
3
Outline
•  Why would you read a paper?
•  What are papers?
•  How to read a paper?
•  Results and evaluation
•  Writing while reading
4
Outline
•  Why would you read a paper?
•  What are papers?
•  How to read a paper?
•  Results and evaluation
•  Writing while reading
5
Why would you read a paper? (1)
•  To know what it is about
◦  superficial reading, skimming
•  To fully assimilate its content
◦  deep reading
•  To find an information you know it contains
◦  a sentence to quote, a figure, a reference
6
Why would you read a paper? (2)
•  To write a reading note about it
◦  private evaluation
•  To review it
◦  for a conference or journal: formal evaluation
◦  for a colleague
•  To include it in a synthesis note
◦  survey
◦  literature review
•  To include it as a reference
◦  in another paper, PhD thesis, grant proposal...
•  To get inspiration
To	write!	 7
Outline
•  Why would you read a paper?
•  What are papers?
•  How to read a paper?
•  Results and evaluation
•  Writing while reading
8
Different kinds of papers
•  Theoretical papers
◦  Theorems, proofs, concepts
◦  e.g. problem solving, algorithm complexity, formal
semantics of a language
•  Empirical papers
◦  Proposal and experimental evaluation
◦  Description and analysis
◦  e.g. of architecture, algorithm, model, software, usages
•  Surveys
◦  Exhaustive state of the art on a subject
◦  e.g. sequence mining
◦  (in other disciplines: meta-review)
•  …
9
What’s in a paper?
Introduc9on	
Body	
Conclusion	and	future	work	
References	
Title	
Abstract	
Keywords	
Author(s)	
Problem	statement	
Comparison	to		
the	literature	
Discussion		
Claim	
Elements	of	valida9on	
10
For a classical
experimental paper
Introduc9on	
Conclusion	and	future	work	
References	
Title	
Abstract	
Keywords	
Author(s)	
Related	works	
System	descrip5on	
Evalua5on	
Results	
Discussion	
11
What is a good paper?
•  Problem is clearly described and interesting (technically
hard, impactful, etc.)
•  Definitions are clear and precise
•  Original contribution is provided compared with related
works
•  Protocol is clearly described, reproducible, satisfying
•  Experimental results that support the propositions
◦  nicely described and analysed
•  Related works are well cited and correctly discussed
•  So, what is a bad paper?
12
Outline
•  Why would you read a paper?
•  What are papers?
•  How to read a paper?
•  Results and evaluation
•  Writing while reading
13
So many papers and still 24 hours in a day
•  Any research field is rich of many papers
◦  several thousands
◦  impossible to read them all
•  Still, necessity to produce scientific knowledge
◦  situated wrt the state of the art
•  Step 1: find articles (see dedicated course)
◦  read title, abstract and keywords to decide if downloading the full
article is needed
•  Step 2: read the articles
◦  but still many articles
14	
Render	of	a	Clock	by	
PoultryChamp	
is	licensed	under	CC	BY	3.0
Active reading
•  Get rid of distractions
•  Get a pen
•  Jump around, re-read, go backward
•  Talk to others
15
A three-pass
approach
1.  Getting a general
idea about the
paper
2.  Grasping the
paper’s content,
but not its
details
3.  Understanding
the paper in
depth
10	minutes	
one	hour	
several	hours	
16	S.	Keshav.	2007.	How	to	read	a	paper.		SIGCOMM	Comput.	Commun.	Rev.	37,	3	(July	2007),	83-84.
Questions to keep in mind
•  What questions does the paper address?
•  Is the problem relevant?
•  What are the main conclusions of the paper?
•  What evidence supports these conclusions?
•  What it the quality of the evidence?
•  Why are the conclusions important?
17
Pass1: overview
•  Title + authors
◦  What is it about + Where does it come from?
•  Abstract
◦  What was done, what is the contribution?
•  Medium
◦  What is the audience?
•  Introduction / conclusion
◦  What are the context, problem, results?
•  Sections/subsections headings (+ glance at formulas & figures)
◦  What is the paper general structure?
•  References
◦  Is it a serious paper?
18
Pass 1: at the end
•  Can I answer these questions?
◦  Category
◦  Context
◦  Correctness
◦  Contributions
◦  Clarity
•  Do I need to go further?
5	Cs	
19
Pass 2: get the author message
•  Identify key parts and articulation of the reasoning
•  Identify key intuitions, limitations, relevance of the work
•  Read carefully the text
•  Focus on figures and their legends
◦  do not spend time on demonstrations
•  Mark down new terms, questions that arise
•  Select references you may would like to consult
20
Pass 2: at the end
•  Am I able to summarize the paper / justify the
results to someone else?
•  Do I need to go further?
21
Pass3: careful reading
•  Go again from beginning to end...
…and jump backwards
•  Identify and challenge every assumption
•  Read proofs
•  Get help from related papers if it gets too
complicated
•  Come back to it later
22
Pass 3: at the end
•  Do I fully understand the work?
•  Can I remake it virtually?
•  What are hidden assumptions?
•  Where would I do things differently?
•  A full-fledged review should go that far
•  A paper read that far should go to the “reference pool” of
one’s work and influence it
•  Living with a paper
◦  Research papers summarize months or years of work in a few
pages à came back, re-read, question, re-discover
23
Potential problems
•  You don’t understand the context and positioning
à read basic references in the domain to situate it
•  You don’t understand the contribution
à if the paper has been accepted there should be one
à look a the related works, confront to the other
approaches
•  You don’t understand the technical details
à do you have to?
à if yes, identify where is the problem to go deeper /
seek help
24
Summary: navigate between
Pass	1	 Pass	2	 Pass	3	
skimming	 deep	reading	
idea	of	the	subject	 good	understanding	
25
Beyond PDF
eLife	LENS	-	hWp://elifesciences.org/elife-news/lens
Outline
•  Why would you read a paper?
•  What are papers?
•  How to read a paper?
•  Results and evaluation
•  Writing while reading
27
Examples of contributions
•  A new way to solve a
problem
•  A new problem
•  The proof of a theorem
•  A data model
•  Indicators of the
behaviour of a system
•  An new implementation of
a known algorithm
•  An interaction principle
•  A language
•  A new concept to
describe usages
•  A collaborative system
•  A methodology for
designing systems
•  A study of how users
behave with a system
Variety	of	contribu9ons	à	Variety	of	jus9fica9ons					
28
Types of justifications
•  Formal proofs
◦  Mathematics, logics
•  Experimentations
◦  Experimental: discovering facts, investigating, testing
hypothesis
◦  Validation: proving that a theory/idea/algorithm/setting
is correct
•  Argumentation
◦  Logical argumentation
•  …
29
Formal Proofs
•  Prove the theoretical complexity of an algorithm
◦  Linear? Exponential?
•  Prove the completeness and correction of an
algorithm
◦  Does it produce all the results?
◦  Does it produce the wanted results?
•  Needs
◦  Formalisation of the data and the algorithm
◦  (use of lemma)
30
Experimentations (1)
•  Check the calculation time for an algorithm
◦  To check if it matches theoretical complexity
◦  To observe its functioning (when too complex)
•  Needs
◦  Precise time measure
◦  Carefully chosen, convincing trial data
◦  Precise evaluation methodology: running trial,
managing result data
◦  Definition of indicators
31
Experimentations (2)
•  Show that a system is more efficient to do a task
than another
•  Needs
◦  Define independent and dependant variables
◦  Make hypothesis
◦  Define protocol
•  e.g. subjects, intra or inter-group
◦  Statistically check null hypothesis
•  p < .05 meaning
32
Experimentations (3)
•  Show the efficiency of an information retrieval
system
•  Needs
◦  Define a corpus of documents
◦  Define some “ground truth”
•  for that query those are the docs that match
◦  Evaluate precision and recall
•  fraction of retrieved docs that are relevant
•  fraction of relevant docs that are retrieved
•  Variations: clustering, learning, etc.
33
Experimentations (4)
•  Check what happens with a new system
•  Check parameter sensitivity for an algorithm on
the quality of results
•  Needs
◦  Build the system
◦  Observe how it behaves
•  E.g. results, memory usage
◦  Draw empirical conclusions
34
Argumentation
•  Validate a model: check whether an ontology is
appropriated for what it was conceived for
•  Needs
◦  Produce characteristics that define appropriateness
◦  Define model or meta-model
◦  Check the result against the characteristics
•  e.g. capacity to describe a domain, simplicity, extensibility
35
Benchmarks
•  General idea
◦  define standard shared datasets against which
various approaches can be compared
◦  mutualisation of the costs of building benchmarks
•  Domains
◦  Learning, clustering, information retrieval, language
recognition, graph matching, etc.
•  Contests
◦  Public learning sets
◦  Test sets used for comparing
◦  e.g. TREC (Text REtrieval Conference)
36
Outline
•  Why would you read a paper?
•  What are papers?
•  How to read a paper?
•  Results and evaluation
•  Writing while reading
37
Active reading
•  Not only sequential reading
◦  skipping parts, getting back, see elsewhere
•  Writing while reading
◦  there is always a writing objective
•  possibly on the long term
•  possibly never to be written
◦  annotating, taking notes is preparing the realization of
that objective
•  No standard method
◦  Very idiosyncratic
38
Annotations
39	Por9on	of	leaf	C4r	by	kladcat		is	licenced	CC	BY	SA	2.0
Annotations
40	Annotated	course	by	Gabriela	Fernandez	is	licenced	CC-BY	4.0
Active reading result (1)
Reading notes
•  Goal
◦  systematically keep track of the content of the papers
that have been read
•  act as a proxy for the article
◦  useful to refresh one’s memory without re-reading the article
•  allow exchange about an article
◦  with one’s supervisor
◦  within one’s research team
•  Difficulty
◦  finding the good granularity
•  personal summary of article + important ideas for one’s
research
◦  be systematic
41
Active reading result (1)
Reading notes
•  Writing reading notes
◦  Structure
•  Bibliographical information
•  Type of article
•  Subject and reason why it has been read
•  Main contributions
•  Summary: problem, solution, experimental setting, results, etc.
•  Commentary
◦  good points, limits
◦  lessons learned with regard to one’s personal research interest
◦  From 1 to 4 pages
•  Could be blogged!
42
Active reading result (2)
Synthesis / literature review
•  Goal: build an exhaustive personal view of a scientific
sub-field
◦  First task of a master’s thesis, a PhD
◦  Will be used
•  in “Related work” sections of articles
•  in the first part of your thesis
•  (in a review article)
•  Difficulty: finding the appropriate articles
◦  Core articles and side articles
◦  Triangulate: spot
•  articles cited by everybody
•  important authors (see webpages)
•  important conferences (skim through program)
43
Active reading result (2)
Synthesis / literature review
•  Writing Synthesis
◦  Organize articles along themes / sections
•  an article can be cited in several themes
•  draw conclusion at the end of each section
◦  When it comes to describing systems
•  summarize in table with characteristics that help to
compare, only describe exemplar systems
◦  Terminate with remarks related to your research
•  Directions you should take
44
Active reading result (3)
Article review
•  Goal
◦  Give your opinion on the acceptance of an article for
publication
•  journal / conference
◦  Give advice on how to improve the article
◦  Most of the time anonymous
•  Difficulty
◦  Takes time
◦  Not easy
•  e.g. assessing interest of a paper vs assessing technical
quality
45
Active reading result (3)
Article review
•  Writing Article reviews
◦  Necessitates deep reading (pass 3)
◦  There are guidelines
◦  Advice
•  Take your time
◦  Read one day, re-read and write the review another day
•  Be careful of what you write (be gentle)
◦  Would you accept to get that comment?
◦  Some reviewers drop anonymity
•  Be ethical
◦  Do not use original material you read while doing a review
◦  Do not accept to review in case of conflict of interest
◦  See http://www.scoop.it/t/toolsandmethodologyforresearch for advice
46

Mais conteúdo relacionado

Mais procurados

Research beyond borders sky is the limit
Research beyond borders sky is the limitResearch beyond borders sky is the limit
Research beyond borders sky is the limitKhalid Hakeem
 
Scholarship of Teaching and Learning - IRB
Scholarship of Teaching and Learning - IRBScholarship of Teaching and Learning - IRB
Scholarship of Teaching and Learning - IRBStaci Trekles
 
Lecture 9.28.10
Lecture 9.28.10Lecture 9.28.10
Lecture 9.28.10VMRoberts
 
IHTE academic publishing strategy example
IHTE academic publishing strategy exampleIHTE academic publishing strategy example
IHTE academic publishing strategy exampleHeli Väätäjä
 
Techniques d’etudes et de recherche
Techniques d’etudes et de rechercheTechniques d’etudes et de recherche
Techniques d’etudes et de rechercheMohamed Diallo
 
Dr Stephen j Hall 2016 Educated in Deep Doctorate Land
Dr Stephen j Hall 2016 Educated in Deep Doctorate Land Dr Stephen j Hall 2016 Educated in Deep Doctorate Land
Dr Stephen j Hall 2016 Educated in Deep Doctorate Land Stephen j Hall
 
Presentation skills for researchers
Presentation skills for researchersPresentation skills for researchers
Presentation skills for researchersManuscriptpedia
 
Publishing and Disseminating your Research and Practice
Publishing and Disseminating your Research and PracticePublishing and Disseminating your Research and Practice
Publishing and Disseminating your Research and PracticeHelen Fallon
 
Potentially predatory and cloned journals february 2022
Potentially predatory and cloned journals february 2022Potentially predatory and cloned journals february 2022
Potentially predatory and cloned journals february 2022Dr. Chinchu C
 
LIASA HELIG Publishing Tips for Librarians
LIASA HELIG Publishing Tips for LibrariansLIASA HELIG Publishing Tips for Librarians
LIASA HELIG Publishing Tips for LibrariansHELIGLIASA
 
PSY 3330 Library and literature searching 2019
PSY 3330  Library and literature searching 2019PSY 3330  Library and literature searching 2019
PSY 3330 Library and literature searching 2019Middlesex University
 
Paper Clinic: Academic Writing and Ethical Practice in Writing 2019
Paper Clinic: Academic Writing and Ethical Practice in Writing 2019Paper Clinic: Academic Writing and Ethical Practice in Writing 2019
Paper Clinic: Academic Writing and Ethical Practice in Writing 2019Dr. Vinod Kumar Kanvaria
 
PSY3330 lecture 2017
PSY3330 lecture 2017PSY3330 lecture 2017
PSY3330 lecture 2017John Iona
 
Presentation EPINOR-lunsj
Presentation EPINOR-lunsjPresentation EPINOR-lunsj
Presentation EPINOR-lunsjEPINOR
 
Literature review in social sciences
Literature review in social sciencesLiterature review in social sciences
Literature review in social sciencesDr. Chinchu C
 
Writing a Successful Paper (Academic Writing Engineering)
Writing a Successful Paper (Academic Writing Engineering)Writing a Successful Paper (Academic Writing Engineering)
Writing a Successful Paper (Academic Writing Engineering)Tarek Gaber
 
Literature gap identification gvk sharma-1
Literature gap identification   gvk   sharma-1Literature gap identification   gvk   sharma-1
Literature gap identification gvk sharma-1Pooja Tripathi
 
Research Paper Boot Camp Webinar Spring 2015
Research Paper Boot Camp Webinar Spring 2015Research Paper Boot Camp Webinar Spring 2015
Research Paper Boot Camp Webinar Spring 2015Bethany Marston, MSEd
 

Mais procurados (20)

Research beyond borders sky is the limit
Research beyond borders sky is the limitResearch beyond borders sky is the limit
Research beyond borders sky is the limit
 
Scholarship of Teaching and Learning - IRB
Scholarship of Teaching and Learning - IRBScholarship of Teaching and Learning - IRB
Scholarship of Teaching and Learning - IRB
 
Lecture 9.28.10
Lecture 9.28.10Lecture 9.28.10
Lecture 9.28.10
 
IHTE academic publishing strategy example
IHTE academic publishing strategy exampleIHTE academic publishing strategy example
IHTE academic publishing strategy example
 
Techniques d’etudes et de recherche
Techniques d’etudes et de rechercheTechniques d’etudes et de recherche
Techniques d’etudes et de recherche
 
Crafting a research agenda
Crafting a research agendaCrafting a research agenda
Crafting a research agenda
 
Dr Stephen j Hall 2016 Educated in Deep Doctorate Land
Dr Stephen j Hall 2016 Educated in Deep Doctorate Land Dr Stephen j Hall 2016 Educated in Deep Doctorate Land
Dr Stephen j Hall 2016 Educated in Deep Doctorate Land
 
Presentation skills for researchers
Presentation skills for researchersPresentation skills for researchers
Presentation skills for researchers
 
Publishing and Disseminating your Research and Practice
Publishing and Disseminating your Research and PracticePublishing and Disseminating your Research and Practice
Publishing and Disseminating your Research and Practice
 
Potentially predatory and cloned journals february 2022
Potentially predatory and cloned journals february 2022Potentially predatory and cloned journals february 2022
Potentially predatory and cloned journals february 2022
 
LIASA HELIG Publishing Tips for Librarians
LIASA HELIG Publishing Tips for LibrariansLIASA HELIG Publishing Tips for Librarians
LIASA HELIG Publishing Tips for Librarians
 
Organizing for Dissertation Success
Organizing for Dissertation SuccessOrganizing for Dissertation Success
Organizing for Dissertation Success
 
PSY 3330 Library and literature searching 2019
PSY 3330  Library and literature searching 2019PSY 3330  Library and literature searching 2019
PSY 3330 Library and literature searching 2019
 
Paper Clinic: Academic Writing and Ethical Practice in Writing 2019
Paper Clinic: Academic Writing and Ethical Practice in Writing 2019Paper Clinic: Academic Writing and Ethical Practice in Writing 2019
Paper Clinic: Academic Writing and Ethical Practice in Writing 2019
 
PSY3330 lecture 2017
PSY3330 lecture 2017PSY3330 lecture 2017
PSY3330 lecture 2017
 
Presentation EPINOR-lunsj
Presentation EPINOR-lunsjPresentation EPINOR-lunsj
Presentation EPINOR-lunsj
 
Literature review in social sciences
Literature review in social sciencesLiterature review in social sciences
Literature review in social sciences
 
Writing a Successful Paper (Academic Writing Engineering)
Writing a Successful Paper (Academic Writing Engineering)Writing a Successful Paper (Academic Writing Engineering)
Writing a Successful Paper (Academic Writing Engineering)
 
Literature gap identification gvk sharma-1
Literature gap identification   gvk   sharma-1Literature gap identification   gvk   sharma-1
Literature gap identification gvk sharma-1
 
Research Paper Boot Camp Webinar Spring 2015
Research Paper Boot Camp Webinar Spring 2015Research Paper Boot Camp Webinar Spring 2015
Research Paper Boot Camp Webinar Spring 2015
 

Destaque

Destaque (8)

Case study final
Case study finalCase study final
Case study final
 
descriptive method
descriptive methoddescriptive method
descriptive method
 
Descriptive Method
Descriptive MethodDescriptive Method
Descriptive Method
 
Descriptive Method
Descriptive MethodDescriptive Method
Descriptive Method
 
Types of descriptive research
Types of descriptive researchTypes of descriptive research
Types of descriptive research
 
Chapter 2:review of related literature and studies
Chapter 2:review of related literature and studiesChapter 2:review of related literature and studies
Chapter 2:review of related literature and studies
 
Descriptive research
Descriptive researchDescriptive research
Descriptive research
 
Types of Research
Types of ResearchTypes of Research
Types of Research
 

Semelhante a Reading Scientific Papers: Active Reading Techniques

Research Methodology and Tips on Better Research
Research Methodology and Tips on Better ResearchResearch Methodology and Tips on Better Research
Research Methodology and Tips on Better ResearchRushdi Shams
 
Literature review in behavioural sciences 10 01 2022
Literature review in behavioural sciences 10 01 2022Literature review in behavioural sciences 10 01 2022
Literature review in behavioural sciences 10 01 2022Dr. Chinchu C
 
Journal selection and fast track publication
Journal selection and fast track publicationJournal selection and fast track publication
Journal selection and fast track publicationMuhammad Imran
 
Journal selection and fast track publication
Journal selection and fast track publicationJournal selection and fast track publication
Journal selection and fast track publicationMuhammad Imran
 
Journal selection and fast track publication
Journal selection and fast track publicationJournal selection and fast track publication
Journal selection and fast track publicationMuhammad Imran
 
Research article structure
Research article structureResearch article structure
Research article structureAhmed Meri
 
Publishing research papers
Publishing research papersPublishing research papers
Publishing research papersVikram Singh
 
CL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdf
CL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdfCL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdf
CL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdfssuserb76cdd
 
MNS Lecture 1.pptx
MNS Lecture 1.pptxMNS Lecture 1.pptx
MNS Lecture 1.pptxShaistaRiaz4
 
Ma dissertation workshop
Ma dissertation workshopMa dissertation workshop
Ma dissertation workshopCarolina Matos
 
Research 101: How to Read a Scientific Paper
Research 101: How to Read a Scientific PaperResearch 101: How to Read a Scientific Paper
Research 101: How to Read a Scientific PaperHarold Gamero
 
Writing a scientific paper
Writing a scientific paperWriting a scientific paper
Writing a scientific paperSylvie Noël
 
Thesis Writing format.pptx
Thesis Writing format.pptxThesis Writing format.pptx
Thesis Writing format.pptxAjit Shinde
 
How to write a thesis
How to write a thesisHow to write a thesis
How to write a thesisArshaps2
 
Publishing a research article malang v2
Publishing a research article malang v2Publishing a research article malang v2
Publishing a research article malang v2Nadidah Safitri
 
UNIT 6.pptx
UNIT 6.pptxUNIT 6.pptx
UNIT 6.pptxsarasiby
 

Semelhante a Reading Scientific Papers: Active Reading Techniques (20)

Research Methodology and Tips on Better Research
Research Methodology and Tips on Better ResearchResearch Methodology and Tips on Better Research
Research Methodology and Tips on Better Research
 
Literature review in behavioural sciences 10 01 2022
Literature review in behavioural sciences 10 01 2022Literature review in behavioural sciences 10 01 2022
Literature review in behavioural sciences 10 01 2022
 
Research from ideas to_publication
Research from ideas to_publicationResearch from ideas to_publication
Research from ideas to_publication
 
Journal selection and fast track publication
Journal selection and fast track publicationJournal selection and fast track publication
Journal selection and fast track publication
 
Journal selection and fast track publication
Journal selection and fast track publicationJournal selection and fast track publication
Journal selection and fast track publication
 
Journal selection and fast track publication
Journal selection and fast track publicationJournal selection and fast track publication
Journal selection and fast track publication
 
Research article structure
Research article structureResearch article structure
Research article structure
 
Publishing research papers
Publishing research papersPublishing research papers
Publishing research papers
 
CL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdf
CL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdfCL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdf
CL7 Selection of Journals Module 4 RPE-Rijo TKMCE.pdf
 
MNS Lecture 1.pptx
MNS Lecture 1.pptxMNS Lecture 1.pptx
MNS Lecture 1.pptx
 
How to write a research paper?
How to write a research paper?How to write a research paper?
How to write a research paper?
 
Ma dissertation workshop
Ma dissertation workshopMa dissertation workshop
Ma dissertation workshop
 
Research 101: How to Read a Scientific Paper
Research 101: How to Read a Scientific PaperResearch 101: How to Read a Scientific Paper
Research 101: How to Read a Scientific Paper
 
Writing a scientific paper
Writing a scientific paperWriting a scientific paper
Writing a scientific paper
 
thesis frame.pptx
thesis frame.pptxthesis frame.pptx
thesis frame.pptx
 
Thesis Writing format.pptx
Thesis Writing format.pptxThesis Writing format.pptx
Thesis Writing format.pptx
 
How to write a thesis
How to write a thesisHow to write a thesis
How to write a thesis
 
Publishing a research article malang v2
Publishing a research article malang v2Publishing a research article malang v2
Publishing a research article malang v2
 
How to write a Great Research Paper?
How to write a Great Research Paper?How to write a Great Research Paper?
How to write a Great Research Paper?
 
UNIT 6.pptx
UNIT 6.pptxUNIT 6.pptx
UNIT 6.pptx
 

Mais de Yannick Prié (Enseignement) (11)

CM processus-unifie
CM processus-unifieCM processus-unifie
CM processus-unifie
 
CM processus agile
CM processus agileCM processus agile
CM processus agile
 
CM Processus Méthodes
CM Processus MéthodesCM Processus Méthodes
CM Processus Méthodes
 
CM patterns
CM patternsCM patterns
CM patterns
 
CM CU-cockburn
CM CU-cockburnCM CU-cockburn
CM CU-cockburn
 
CM uml-intro
CM uml-introCM uml-intro
CM uml-intro
 
CM uml-diag-statiques
CM uml-diag-statiquesCM uml-diag-statiques
CM uml-diag-statiques
 
CM uml-diag-dynamiques-interaction
CM uml-diag-dynamiques-interactionCM uml-diag-dynamiques-interaction
CM uml-diag-dynamiques-interaction
 
CM uml-concepts-avances
CM uml-concepts-avancesCM uml-concepts-avances
CM uml-concepts-avances
 
CM CSI
CM CSICM CSI
CM CSI
 
CM rappels-objet
CM rappels-objetCM rappels-objet
CM rappels-objet
 

Último

Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Celine George
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxpboyjonauth
 
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxContemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxRoyAbrique
 
Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...
Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...
Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...RKavithamani
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformChameera Dedduwage
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxheathfieldcps1
 
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdfssuser54595a
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphThiyagu K
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)eniolaolutunde
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdfQucHHunhnh
 
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxSOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxiammrhaywood
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxSayali Powar
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfsanyamsingh5019
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpinRaunakKeshri1
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfchloefrazer622
 

Último (20)

Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptx
 
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxContemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
 
Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...
Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...
Privatization and Disinvestment - Meaning, Objectives, Advantages and Disadva...
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy Reform
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
 
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
 
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxSOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdf
 
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpin
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
 
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptxINDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdf
 

Reading Scientific Papers: Active Reading Techniques

  • 1. Methodology and Tools for Research: Reading scientific material Yannick Prié Polytech Nantes, University of Nantes Master DMKM, 2015-2016 CC BY-SA 4.0
  • 2. Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) •  This course "Methodology and Tools for Research: Reading Scientific Material" by Yannick Prié is licensed CC BY-SA 4.0 •  This license covers the general organization of the material, the textual content, the figures, etc. except where indicated. •  This license means that you can share and adapt this course, provided you give appropriate credit to the author and distribute your contributions under the same license as the original ◦  for more information about this license, see http://creativecommons.org/licenses/by-sa/4.0/
  • 3. Objectives of this course •  How to read a paper? •  Various kinds of validations •  Kinds of writings related to reading ◦  Active reading and annotations ◦  Reading notes ◦  Synthesis ◦  Reviews 3
  • 4. Outline •  Why would you read a paper? •  What are papers? •  How to read a paper? •  Results and evaluation •  Writing while reading 4
  • 5. Outline •  Why would you read a paper? •  What are papers? •  How to read a paper? •  Results and evaluation •  Writing while reading 5
  • 6. Why would you read a paper? (1) •  To know what it is about ◦  superficial reading, skimming •  To fully assimilate its content ◦  deep reading •  To find an information you know it contains ◦  a sentence to quote, a figure, a reference 6
  • 7. Why would you read a paper? (2) •  To write a reading note about it ◦  private evaluation •  To review it ◦  for a conference or journal: formal evaluation ◦  for a colleague •  To include it in a synthesis note ◦  survey ◦  literature review •  To include it as a reference ◦  in another paper, PhD thesis, grant proposal... •  To get inspiration To write! 7
  • 8. Outline •  Why would you read a paper? •  What are papers? •  How to read a paper? •  Results and evaluation •  Writing while reading 8
  • 9. Different kinds of papers •  Theoretical papers ◦  Theorems, proofs, concepts ◦  e.g. problem solving, algorithm complexity, formal semantics of a language •  Empirical papers ◦  Proposal and experimental evaluation ◦  Description and analysis ◦  e.g. of architecture, algorithm, model, software, usages •  Surveys ◦  Exhaustive state of the art on a subject ◦  e.g. sequence mining ◦  (in other disciplines: meta-review) •  … 9
  • 10. What’s in a paper? Introduc9on Body Conclusion and future work References Title Abstract Keywords Author(s) Problem statement Comparison to the literature Discussion Claim Elements of valida9on 10
  • 11. For a classical experimental paper Introduc9on Conclusion and future work References Title Abstract Keywords Author(s) Related works System descrip5on Evalua5on Results Discussion 11
  • 12. What is a good paper? •  Problem is clearly described and interesting (technically hard, impactful, etc.) •  Definitions are clear and precise •  Original contribution is provided compared with related works •  Protocol is clearly described, reproducible, satisfying •  Experimental results that support the propositions ◦  nicely described and analysed •  Related works are well cited and correctly discussed •  So, what is a bad paper? 12
  • 13. Outline •  Why would you read a paper? •  What are papers? •  How to read a paper? •  Results and evaluation •  Writing while reading 13
  • 14. So many papers and still 24 hours in a day •  Any research field is rich of many papers ◦  several thousands ◦  impossible to read them all •  Still, necessity to produce scientific knowledge ◦  situated wrt the state of the art •  Step 1: find articles (see dedicated course) ◦  read title, abstract and keywords to decide if downloading the full article is needed •  Step 2: read the articles ◦  but still many articles 14 Render of a Clock by PoultryChamp is licensed under CC BY 3.0
  • 15. Active reading •  Get rid of distractions •  Get a pen •  Jump around, re-read, go backward •  Talk to others 15
  • 16. A three-pass approach 1.  Getting a general idea about the paper 2.  Grasping the paper’s content, but not its details 3.  Understanding the paper in depth 10 minutes one hour several hours 16 S. Keshav. 2007. How to read a paper. SIGCOMM Comput. Commun. Rev. 37, 3 (July 2007), 83-84.
  • 17. Questions to keep in mind •  What questions does the paper address? •  Is the problem relevant? •  What are the main conclusions of the paper? •  What evidence supports these conclusions? •  What it the quality of the evidence? •  Why are the conclusions important? 17
  • 18. Pass1: overview •  Title + authors ◦  What is it about + Where does it come from? •  Abstract ◦  What was done, what is the contribution? •  Medium ◦  What is the audience? •  Introduction / conclusion ◦  What are the context, problem, results? •  Sections/subsections headings (+ glance at formulas & figures) ◦  What is the paper general structure? •  References ◦  Is it a serious paper? 18
  • 19. Pass 1: at the end •  Can I answer these questions? ◦  Category ◦  Context ◦  Correctness ◦  Contributions ◦  Clarity •  Do I need to go further? 5 Cs 19
  • 20. Pass 2: get the author message •  Identify key parts and articulation of the reasoning •  Identify key intuitions, limitations, relevance of the work •  Read carefully the text •  Focus on figures and their legends ◦  do not spend time on demonstrations •  Mark down new terms, questions that arise •  Select references you may would like to consult 20
  • 21. Pass 2: at the end •  Am I able to summarize the paper / justify the results to someone else? •  Do I need to go further? 21
  • 22. Pass3: careful reading •  Go again from beginning to end... …and jump backwards •  Identify and challenge every assumption •  Read proofs •  Get help from related papers if it gets too complicated •  Come back to it later 22
  • 23. Pass 3: at the end •  Do I fully understand the work? •  Can I remake it virtually? •  What are hidden assumptions? •  Where would I do things differently? •  A full-fledged review should go that far •  A paper read that far should go to the “reference pool” of one’s work and influence it •  Living with a paper ◦  Research papers summarize months or years of work in a few pages à came back, re-read, question, re-discover 23
  • 24. Potential problems •  You don’t understand the context and positioning à read basic references in the domain to situate it •  You don’t understand the contribution à if the paper has been accepted there should be one à look a the related works, confront to the other approaches •  You don’t understand the technical details à do you have to? à if yes, identify where is the problem to go deeper / seek help 24
  • 25. Summary: navigate between Pass 1 Pass 2 Pass 3 skimming deep reading idea of the subject good understanding 25
  • 27. Outline •  Why would you read a paper? •  What are papers? •  How to read a paper? •  Results and evaluation •  Writing while reading 27
  • 28. Examples of contributions •  A new way to solve a problem •  A new problem •  The proof of a theorem •  A data model •  Indicators of the behaviour of a system •  An new implementation of a known algorithm •  An interaction principle •  A language •  A new concept to describe usages •  A collaborative system •  A methodology for designing systems •  A study of how users behave with a system Variety of contribu9ons à Variety of jus9fica9ons 28
  • 29. Types of justifications •  Formal proofs ◦  Mathematics, logics •  Experimentations ◦  Experimental: discovering facts, investigating, testing hypothesis ◦  Validation: proving that a theory/idea/algorithm/setting is correct •  Argumentation ◦  Logical argumentation •  … 29
  • 30. Formal Proofs •  Prove the theoretical complexity of an algorithm ◦  Linear? Exponential? •  Prove the completeness and correction of an algorithm ◦  Does it produce all the results? ◦  Does it produce the wanted results? •  Needs ◦  Formalisation of the data and the algorithm ◦  (use of lemma) 30
  • 31. Experimentations (1) •  Check the calculation time for an algorithm ◦  To check if it matches theoretical complexity ◦  To observe its functioning (when too complex) •  Needs ◦  Precise time measure ◦  Carefully chosen, convincing trial data ◦  Precise evaluation methodology: running trial, managing result data ◦  Definition of indicators 31
  • 32. Experimentations (2) •  Show that a system is more efficient to do a task than another •  Needs ◦  Define independent and dependant variables ◦  Make hypothesis ◦  Define protocol •  e.g. subjects, intra or inter-group ◦  Statistically check null hypothesis •  p < .05 meaning 32
  • 33. Experimentations (3) •  Show the efficiency of an information retrieval system •  Needs ◦  Define a corpus of documents ◦  Define some “ground truth” •  for that query those are the docs that match ◦  Evaluate precision and recall •  fraction of retrieved docs that are relevant •  fraction of relevant docs that are retrieved •  Variations: clustering, learning, etc. 33
  • 34. Experimentations (4) •  Check what happens with a new system •  Check parameter sensitivity for an algorithm on the quality of results •  Needs ◦  Build the system ◦  Observe how it behaves •  E.g. results, memory usage ◦  Draw empirical conclusions 34
  • 35. Argumentation •  Validate a model: check whether an ontology is appropriated for what it was conceived for •  Needs ◦  Produce characteristics that define appropriateness ◦  Define model or meta-model ◦  Check the result against the characteristics •  e.g. capacity to describe a domain, simplicity, extensibility 35
  • 36. Benchmarks •  General idea ◦  define standard shared datasets against which various approaches can be compared ◦  mutualisation of the costs of building benchmarks •  Domains ◦  Learning, clustering, information retrieval, language recognition, graph matching, etc. •  Contests ◦  Public learning sets ◦  Test sets used for comparing ◦  e.g. TREC (Text REtrieval Conference) 36
  • 37. Outline •  Why would you read a paper? •  What are papers? •  How to read a paper? •  Results and evaluation •  Writing while reading 37
  • 38. Active reading •  Not only sequential reading ◦  skipping parts, getting back, see elsewhere •  Writing while reading ◦  there is always a writing objective •  possibly on the long term •  possibly never to be written ◦  annotating, taking notes is preparing the realization of that objective •  No standard method ◦  Very idiosyncratic 38
  • 41. Active reading result (1) Reading notes •  Goal ◦  systematically keep track of the content of the papers that have been read •  act as a proxy for the article ◦  useful to refresh one’s memory without re-reading the article •  allow exchange about an article ◦  with one’s supervisor ◦  within one’s research team •  Difficulty ◦  finding the good granularity •  personal summary of article + important ideas for one’s research ◦  be systematic 41
  • 42. Active reading result (1) Reading notes •  Writing reading notes ◦  Structure •  Bibliographical information •  Type of article •  Subject and reason why it has been read •  Main contributions •  Summary: problem, solution, experimental setting, results, etc. •  Commentary ◦  good points, limits ◦  lessons learned with regard to one’s personal research interest ◦  From 1 to 4 pages •  Could be blogged! 42
  • 43. Active reading result (2) Synthesis / literature review •  Goal: build an exhaustive personal view of a scientific sub-field ◦  First task of a master’s thesis, a PhD ◦  Will be used •  in “Related work” sections of articles •  in the first part of your thesis •  (in a review article) •  Difficulty: finding the appropriate articles ◦  Core articles and side articles ◦  Triangulate: spot •  articles cited by everybody •  important authors (see webpages) •  important conferences (skim through program) 43
  • 44. Active reading result (2) Synthesis / literature review •  Writing Synthesis ◦  Organize articles along themes / sections •  an article can be cited in several themes •  draw conclusion at the end of each section ◦  When it comes to describing systems •  summarize in table with characteristics that help to compare, only describe exemplar systems ◦  Terminate with remarks related to your research •  Directions you should take 44
  • 45. Active reading result (3) Article review •  Goal ◦  Give your opinion on the acceptance of an article for publication •  journal / conference ◦  Give advice on how to improve the article ◦  Most of the time anonymous •  Difficulty ◦  Takes time ◦  Not easy •  e.g. assessing interest of a paper vs assessing technical quality 45
  • 46. Active reading result (3) Article review •  Writing Article reviews ◦  Necessitates deep reading (pass 3) ◦  There are guidelines ◦  Advice •  Take your time ◦  Read one day, re-read and write the review another day •  Be careful of what you write (be gentle) ◦  Would you accept to get that comment? ◦  Some reviewers drop anonymity •  Be ethical ◦  Do not use original material you read while doing a review ◦  Do not accept to review in case of conflict of interest ◦  See http://www.scoop.it/t/toolsandmethodologyforresearch for advice 46