SlideShare uma empresa Scribd logo
1 de 25
Baixar para ler offline
Natural Language Processing

                  Daniel Dahlmeier

NUS Graduate School for Integrative Sciences and Engineering
              danielhe@comp.nus.edu.sg


            CSTalks 2 November 2011
Acknowledgments




  Examples and figures from Michael Collins’ lecture notes:
  http://www.cs.columbia.edu/∼mcollins.


  Some other figures are from Wikipedia: http://www.wikipedia.org.


  The rest I randomly found on the web.
Examples
                    What is NLP?
                     Background
                       NLP tasks
                   Why is it hard?
                    Related Stuff
                      Conclusion



Google translate




                                     3/25
Examples
                     What is NLP?
                      Background
                        NLP tasks
                    Why is it hard?
                     Related Stuff
                       Conclusion



IBM’s Watson computer wins at Jeopardy!




                                          4/25
Examples
        What is NLP?
         Background
           NLP tasks
       Why is it hard?
        Related Stuff
          Conclusion



Siri




                         5/25
Examples
                             What is NLP?
                              Background
                                NLP tasks
                            Why is it hard?
                             Related Stuff
                               Conclusion



What is Natural Language Processing?


   Natural Language Processing (NLP) or Computational Linguistics
   Language processing that goes beyond a “bag of words” representation.

   Example
       Translate from one language into the other.
       Answer natural language questions.
       Parse the syntactic/semantic structure of a sentence.

   The other NLP
   NLP = neuro-linguistic programming.


                                                                           6/25
Examples
                                What is NLP?
                                 Background
                                   NLP tasks
                               Why is it hard?
                                Related Stuff
                                  Conclusion



Background(s): Artificial Intelligence




   Talk to your computer
       Dave: Hello, HAL. Do you read me, HAL?
       HAL: Affirmative, Dave. I read you.
       Dave: Open the pod bay doors, HAL.
       HAL: I’m sorry, Dave. I’m afraid I can’t do that.

   The computer needs to ...
       Understand the user : Natural Language Understanding.
       Generate a well-formed reply : Natural Language Generation.
                                                                     7/25
Examples
                              What is NLP?
                               Background
                                 NLP tasks
                             Why is it hard?
                              Related Stuff
                                Conclusion



Background(s): Artificial Intelligence (cont.)




   Turing Test
       Experimenter talks to two parties A and B via a terminal.
       If C cannot distinguish which party is a computer and which is a
       human, we should consider the computer to be intelligent.
       Natural language is deeply intertwined with intelligence.
                                                                          8/25
Examples
                              What is NLP?
                               Background
                                 NLP tasks
                             Why is it hard?
                              Related Stuff
                                Conclusion



Background(s): Linguistics




   Generative Linguistics
       Humans can produce and understand an infinite number of
       sentences by means of a finite set of rules.
       Language is produced through a generative, recursive process in the
       human brain.
       The principles that underlie this process are universal to all
       languages (universal grammar).                                        9/25
Examples
                               What is NLP?
                                Background
                                  NLP tasks
                              Why is it hard?
                               Related Stuff
                                 Conclusion



Background(s): the Web



       “We are drowning in information but starved for knowledge.”
       by Edward Osborne Wilson

   Too much text to read...
       Wikipedia: over 3.7 million articles (English).
       PubMed: over 20 million citations.
       WWW: billions of pages, trillions of words.




                                                                     10/25
Examples
                              What is NLP?
                               Background
                                 NLP tasks
                             Why is it hard?
                              Related Stuff
                                Conclusion



Part-of-speech Tagging



   Part-of-speech tagging
       Input: a sentence.
       Output: a part-of-speech tag sequence, e.g., noun, verb, adjective,...

   Example
   Profits/N soared/V at/P Boeing/N Co./N ,/, easily/ADV topping/V
   forecasts/N on/P Wall/N Street/N ./.




                                                                                11/25
Examples
                               What is NLP?
                                Background
                                  NLP tasks
                              Why is it hard?
                               Related Stuff
                                 Conclusion



Named-entity recognition


   Named-entity recognition
       Input: a sentence.
       Output: a BIO-named entity tag sequence, e.g., PERSON,
       ORGANIZATION, OTHER.

   Example
   Profits/O soared/O at/O Boeing/B-ORG Co./I-ORG ,/O easily/O
   topping/O forecasts/O on/O Wall/O Street/O ./O




                                                                12/25
Examples
                             What is NLP?
                              Background
                                NLP tasks
                            Why is it hard?
                             Related Stuff
                               Conclusion



Word Sense Disambiguation



   Word sense disambiguation
       Input: a sentence.
       Output: the sense of each word in the sentence.

   Example
   I/sense1 can/sense1 can/sense2 a/sense1 can sense3 .




                                                          13/25
Examples
                               What is NLP?
                                Background
                                  NLP tasks
                              Why is it hard?
                               Related Stuff
                                 Conclusion



Parsing
   Parsing
       Input: a sentence.
       Output: the syntactic tree structure of the sentence.

   Example
   Boeing is located in Seattle.




                                                               14/25
Examples
                              What is NLP?
                               Background
                                 NLP tasks
                             Why is it hard?
                              Related Stuff
                                Conclusion



Machine translation


   Machine Translation
      Input: a sentence in language F .
       Output: the translated sentence in language E .

   Example
   Input: Syriens Pr¨sident Baschar al-Assad hat den Westen davor
                     a
   gewarnt, sich in die Angelegenheiten seines Landes einzumischen.

   Output: Syrian President Bashar al-Assad has warned the West against
   interfering in the affairs of his country.



                                                                          15/25
Examples
                           What is NLP?
                            Background
                              NLP tasks
                          Why is it hard?
                           Related Stuff
                             Conclusion



Why is it hard? ( example from L.Lee)




       “At last, a computer that understands you like your mother”




                                                                     16/25
Examples
                               What is NLP?
                                Background
                                  NLP tasks
                              Why is it hard?
                               Related Stuff
                                 Conclusion



Ambiguity of Natural Language



          “At last, a computer that understands you like your mother”

   This could mean...
     1   It understands you as well as your mother understands you.
     2   It understands (that) you like your mother.
     3   It understands you as well as it understands your mother.
   1 and 3: Does this mean well, or poorly?




                                                                        17/25
Examples
                               What is NLP?
                                Background
                                  NLP tasks
                              Why is it hard?
                               Related Stuff
                                 Conclusion



Ambiguity at the Acoustic Level




          “At last, a computer that understands you like your mother”

   This sounds like...
     1   “... a computer that understands you like your mother.”
     2   “... a computer that understands you lie cured mother.”




                                                                        18/25
Examples
                           What is NLP?
                            Background
                              NLP tasks
                          Why is it hard?
                           Related Stuff
                             Conclusion



Ambiguity at the Syntactic (structure) Level



       “At last, a computer that understands you like your mother”




                                                                     19/25
Examples
                        What is NLP?
                         Background
                           NLP tasks
                       Why is it hard?
                        Related Stuff
                          Conclusion



Ambiguity at the Syntactic (structure) Level
                   “List all flights on Tuesday.”




                                                   20/25
Examples
                                What is NLP?
                                 Background
                                   NLP tasks
                               Why is it hard?
                                Related Stuff
                                  Conclusion



Ambiguity at the Semantic (meaning) Level


   Definition of “mother”
     1   a woman who has given birth to a child
     2   a stringy slimy substance consisting of yeast cells and bacteria; is
         added to cider or wine to produce vinegar.

   More ambiguity
         They put money in the bank (= buried in mud?).
         I saw her duck with a telescope (= a duck carrying a telescope?).




                                                                                21/25
Examples
                              What is NLP?
                               Background
                                 NLP tasks
                             Why is it hard?
                              Related Stuff
                                Conclusion



Ambiguity at the Discourse (multi-clause) Level



   Anaphora resolution
   Alice says they’ve built a computer that understands you like your
   mother.
   But she ...
       ... doesn’t know any details (Alice)
       ... doesn’t understand me at all (my mother)




                                                                        22/25
Examples
                               What is NLP?
                                Background
                                  NLP tasks
                              Why is it hard?
                               Related Stuff
                                 Conclusion



Related Stuff

   Machine Learning
        This really made large-scale, open domain NLP applications possible.

   Information Retrieval
        Both need to “understand” language.

   Linguistics
        Interested in the nature of language.

   Psychology / Cognitive Science
        Both interested in human cognitive capabilities.


                                                                               23/25
Examples
                              What is NLP?
                               Background
                                 NLP tasks
                             Why is it hard?
                              Related Stuff
                                Conclusion



Conclusion


   What I have told you...
       What NLP is about.
       Some NLP tasks that people work on.
       Why it’s not that easy.

   What I haven’t told you
       How do you solve all these problems?
       How well does it work?
       What is left to be done?



                                               24/25
Examples
                            What is NLP?
                             Background
                               NLP tasks
                           Why is it hard?
                            Related Stuff
                              Conclusion



Would you like to know more?

   NLP courses at NUS
      CS4248: natural language processing
       CS6207: advanced natural language processing

   Books




   Jurafsky and Martin, Speech and Language Processing (2nd Edition)


                                                                       25/25

Mais conteúdo relacionado

Mais procurados

Pal gov.tutorial4.session1 2.whatisontology
Pal gov.tutorial4.session1 2.whatisontologyPal gov.tutorial4.session1 2.whatisontology
Pal gov.tutorial4.session1 2.whatisontologyMustafa Jarrar
 
Pal gov.tutorial4.session8 2.stepwisemethodologies
Pal gov.tutorial4.session8 2.stepwisemethodologiesPal gov.tutorial4.session8 2.stepwisemethodologies
Pal gov.tutorial4.session8 2.stepwisemethodologiesMustafa Jarrar
 
Pal gov.tutorial4.session5.lab ontologytools
Pal gov.tutorial4.session5.lab ontologytoolsPal gov.tutorial4.session5.lab ontologytools
Pal gov.tutorial4.session5.lab ontologytoolsMustafa Jarrar
 
Pal gov.tutorial4.session8 1.ontologymodelingchallenges
Pal gov.tutorial4.session8 1.ontologymodelingchallengesPal gov.tutorial4.session8 1.ontologymodelingchallenges
Pal gov.tutorial4.session8 1.ontologymodelingchallengesMustafa Jarrar
 
Pal gov.tutorial4.session3.lab bankcustomerontology
Pal gov.tutorial4.session3.lab bankcustomerontologyPal gov.tutorial4.session3.lab bankcustomerontology
Pal gov.tutorial4.session3.lab bankcustomerontologyMustafa Jarrar
 
Pal gov.tutorial4.session2.lab populationontology
Pal gov.tutorial4.session2.lab populationontologyPal gov.tutorial4.session2.lab populationontology
Pal gov.tutorial4.session2.lab populationontologyMustafa Jarrar
 
Pal gov.tutorial4.outline
Pal gov.tutorial4.outlinePal gov.tutorial4.outline
Pal gov.tutorial4.outlineMustafa Jarrar
 
Respeaking as a part of translation and interpreting curriculum
Respeaking as a part of translation and interpreting curriculumRespeaking as a part of translation and interpreting curriculum
Respeaking as a part of translation and interpreting curriculumUniversity of Warsaw
 
Pal gov.tutorial4.session6 1.ontologyengineeringchallenges
Pal gov.tutorial4.session6 1.ontologyengineeringchallengesPal gov.tutorial4.session6 1.ontologyengineeringchallenges
Pal gov.tutorial4.session6 1.ontologyengineeringchallengesMustafa Jarrar
 
Deep learning for natural language embeddings
Deep learning for natural language embeddingsDeep learning for natural language embeddings
Deep learning for natural language embeddingsRoelof Pieters
 
Practical Deep Learning for NLP
Practical Deep Learning for NLP Practical Deep Learning for NLP
Practical Deep Learning for NLP Textkernel
 
Eurecom and Aalto University at Mediaeval 2021
Eurecom and Aalto University at Mediaeval 2021Eurecom and Aalto University at Mediaeval 2021
Eurecom and Aalto University at Mediaeval 2021Alison Reboud
 
Pal gov.tutorial4.session14 rootinglegalpersonontology
Pal gov.tutorial4.session14 rootinglegalpersonontologyPal gov.tutorial4.session14 rootinglegalpersonontology
Pal gov.tutorial4.session14 rootinglegalpersonontologyMustafa Jarrar
 
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...Christoph Lange
 
Pal gov.tutorial4.session1 1.needforsharedsemantics
Pal gov.tutorial4.session1 1.needforsharedsemanticsPal gov.tutorial4.session1 1.needforsharedsemantics
Pal gov.tutorial4.session1 1.needforsharedsemanticsMustafa Jarrar
 
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...Christoph Lange
 
KiwiPyCon 2014 talk - Understanding human language with Python
KiwiPyCon 2014 talk - Understanding human language with PythonKiwiPyCon 2014 talk - Understanding human language with Python
KiwiPyCon 2014 talk - Understanding human language with PythonAlyona Medelyan
 
DPC2007 Objects Of Desire (Kevlin Henney)
DPC2007 Objects Of Desire (Kevlin Henney)DPC2007 Objects Of Desire (Kevlin Henney)
DPC2007 Objects Of Desire (Kevlin Henney)dpc
 
Deep Learning for Information Retrieval
Deep Learning for Information RetrievalDeep Learning for Information Retrieval
Deep Learning for Information RetrievalRoelof Pieters
 

Mais procurados (20)

Pal gov.tutorial4.session1 2.whatisontology
Pal gov.tutorial4.session1 2.whatisontologyPal gov.tutorial4.session1 2.whatisontology
Pal gov.tutorial4.session1 2.whatisontology
 
Pal gov.tutorial4.session8 2.stepwisemethodologies
Pal gov.tutorial4.session8 2.stepwisemethodologiesPal gov.tutorial4.session8 2.stepwisemethodologies
Pal gov.tutorial4.session8 2.stepwisemethodologies
 
Pal gov.tutorial4.session5.lab ontologytools
Pal gov.tutorial4.session5.lab ontologytoolsPal gov.tutorial4.session5.lab ontologytools
Pal gov.tutorial4.session5.lab ontologytools
 
Pal gov.tutorial4.session8 1.ontologymodelingchallenges
Pal gov.tutorial4.session8 1.ontologymodelingchallengesPal gov.tutorial4.session8 1.ontologymodelingchallenges
Pal gov.tutorial4.session8 1.ontologymodelingchallenges
 
Pal gov.tutorial4.session3.lab bankcustomerontology
Pal gov.tutorial4.session3.lab bankcustomerontologyPal gov.tutorial4.session3.lab bankcustomerontology
Pal gov.tutorial4.session3.lab bankcustomerontology
 
Pal gov.tutorial4.session2.lab populationontology
Pal gov.tutorial4.session2.lab populationontologyPal gov.tutorial4.session2.lab populationontology
Pal gov.tutorial4.session2.lab populationontology
 
Pal gov.tutorial4.outline
Pal gov.tutorial4.outlinePal gov.tutorial4.outline
Pal gov.tutorial4.outline
 
Respeaking as a part of translation and interpreting curriculum
Respeaking as a part of translation and interpreting curriculumRespeaking as a part of translation and interpreting curriculum
Respeaking as a part of translation and interpreting curriculum
 
Pal gov.tutorial4.session6 1.ontologyengineeringchallenges
Pal gov.tutorial4.session6 1.ontologyengineeringchallengesPal gov.tutorial4.session6 1.ontologyengineeringchallenges
Pal gov.tutorial4.session6 1.ontologyengineeringchallenges
 
Deep learning for natural language embeddings
Deep learning for natural language embeddingsDeep learning for natural language embeddings
Deep learning for natural language embeddings
 
Practical Deep Learning for NLP
Practical Deep Learning for NLP Practical Deep Learning for NLP
Practical Deep Learning for NLP
 
Eurecom and Aalto University at Mediaeval 2021
Eurecom and Aalto University at Mediaeval 2021Eurecom and Aalto University at Mediaeval 2021
Eurecom and Aalto University at Mediaeval 2021
 
Pal gov.tutorial4.session14 rootinglegalpersonontology
Pal gov.tutorial4.session14 rootinglegalpersonontologyPal gov.tutorial4.session14 rootinglegalpersonontology
Pal gov.tutorial4.session14 rootinglegalpersonontology
 
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
 
Pal gov.tutorial4.session1 1.needforsharedsemantics
Pal gov.tutorial4.session1 1.needforsharedsemanticsPal gov.tutorial4.session1 1.needforsharedsemantics
Pal gov.tutorial4.session1 1.needforsharedsemantics
 
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
Ontology Integration and Interoperability (OntoIOp) – Part 1: The Distributed...
 
BERT
BERTBERT
BERT
 
KiwiPyCon 2014 talk - Understanding human language with Python
KiwiPyCon 2014 talk - Understanding human language with PythonKiwiPyCon 2014 talk - Understanding human language with Python
KiwiPyCon 2014 talk - Understanding human language with Python
 
DPC2007 Objects Of Desire (Kevlin Henney)
DPC2007 Objects Of Desire (Kevlin Henney)DPC2007 Objects Of Desire (Kevlin Henney)
DPC2007 Objects Of Desire (Kevlin Henney)
 
Deep Learning for Information Retrieval
Deep Learning for Information RetrievalDeep Learning for Information Retrieval
Deep Learning for Information Retrieval
 

Destaque

CSTalks - Named Data Networks - 9 Feb
CSTalks - Named Data Networks - 9 FebCSTalks - Named Data Networks - 9 Feb
CSTalks - Named Data Networks - 9 Febcstalks
 
CSTalks-Natural Language Processing-17Aug
CSTalks-Natural Language Processing-17AugCSTalks-Natural Language Processing-17Aug
CSTalks-Natural Language Processing-17Augcstalks
 
(Paper Presentation) ZIGZAG: An Efficient Peer-to-Peer Scheme for Media Strea...
(Paper Presentation)ZIGZAG: An Efficient Peer-to-Peer Scheme forMedia Strea...(Paper Presentation)ZIGZAG: An Efficient Peer-to-Peer Scheme forMedia Strea...
(Paper Presentation) ZIGZAG: An Efficient Peer-to-Peer Scheme for Media Strea...Rajesh Piryani
 
CSTalks - Model Checking - 26 Jan
CSTalks - Model Checking - 26 JanCSTalks - Model Checking - 26 Jan
CSTalks - Model Checking - 26 Jancstalks
 
CSTalks - Peer-to-peer - 16 Feb
CSTalks - Peer-to-peer - 16 FebCSTalks - Peer-to-peer - 16 Feb
CSTalks - Peer-to-peer - 16 Febcstalks
 
CSTalks - GPGPU - 19 Jan
CSTalks  -  GPGPU - 19 JanCSTalks  -  GPGPU - 19 Jan
CSTalks - GPGPU - 19 Jancstalks
 

Destaque (7)

CSTalks - Named Data Networks - 9 Feb
CSTalks - Named Data Networks - 9 FebCSTalks - Named Data Networks - 9 Feb
CSTalks - Named Data Networks - 9 Feb
 
CSTalks-Natural Language Processing-17Aug
CSTalks-Natural Language Processing-17AugCSTalks-Natural Language Processing-17Aug
CSTalks-Natural Language Processing-17Aug
 
(Paper Presentation) ZIGZAG: An Efficient Peer-to-Peer Scheme for Media Strea...
(Paper Presentation)ZIGZAG: An Efficient Peer-to-Peer Scheme forMedia Strea...(Paper Presentation)ZIGZAG: An Efficient Peer-to-Peer Scheme forMedia Strea...
(Paper Presentation) ZIGZAG: An Efficient Peer-to-Peer Scheme for Media Strea...
 
CSTalks - Model Checking - 26 Jan
CSTalks - Model Checking - 26 JanCSTalks - Model Checking - 26 Jan
CSTalks - Model Checking - 26 Jan
 
CSTalks - Peer-to-peer - 16 Feb
CSTalks - Peer-to-peer - 16 FebCSTalks - Peer-to-peer - 16 Feb
CSTalks - Peer-to-peer - 16 Feb
 
BMC 2012
BMC 2012BMC 2012
BMC 2012
 
CSTalks - GPGPU - 19 Jan
CSTalks  -  GPGPU - 19 JanCSTalks  -  GPGPU - 19 Jan
CSTalks - GPGPU - 19 Jan
 

Semelhante a CSTalks-Natural Language Processing-2 Nov

Portuguese Linguistic Tools: What, Why and How
Portuguese Linguistic Tools: What, Why and HowPortuguese Linguistic Tools: What, Why and How
Portuguese Linguistic Tools: What, Why and HowValeria de Paiva
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processingpunedevscom
 
Testing speaking
Testing speakingTesting speaking
Testing speakingM B
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingParrotAI
 
Natural language processing
Natural language processingNatural language processing
Natural language processingKarenVacca
 
Natural language processing for requirements engineering: ICSE 2021 Technical...
Natural language processing for requirements engineering: ICSE 2021 Technical...Natural language processing for requirements engineering: ICSE 2021 Technical...
Natural language processing for requirements engineering: ICSE 2021 Technical...alessio_ferrari
 
Transfer_Learning_for_Natural_Language_P_v3_MEAP.pdf
Transfer_Learning_for_Natural_Language_P_v3_MEAP.pdfTransfer_Learning_for_Natural_Language_P_v3_MEAP.pdf
Transfer_Learning_for_Natural_Language_P_v3_MEAP.pdforanisalcani
 
Pycon India 2018 Natural Language Processing Workshop
Pycon India 2018   Natural Language Processing WorkshopPycon India 2018   Natural Language Processing Workshop
Pycon India 2018 Natural Language Processing WorkshopLakshya Sivaramakrishnan
 
13. Constantin Orasan (UoW) Natural Language Processing for Translation
13. Constantin Orasan (UoW) Natural Language Processing for Translation13. Constantin Orasan (UoW) Natural Language Processing for Translation
13. Constantin Orasan (UoW) Natural Language Processing for TranslationRIILP
 
ICDM 2019 Tutorial: Speech and Language Processing: New Tools and Applications
ICDM 2019 Tutorial: Speech and Language Processing: New Tools and ApplicationsICDM 2019 Tutorial: Speech and Language Processing: New Tools and Applications
ICDM 2019 Tutorial: Speech and Language Processing: New Tools and ApplicationsForward Gradient
 
Yves Peirsman - Deep Learning for NLP
Yves Peirsman - Deep Learning for NLPYves Peirsman - Deep Learning for NLP
Yves Peirsman - Deep Learning for NLPHendrik D'Oosterlinck
 
Wired_JSpring_08062022.pptx
Wired_JSpring_08062022.pptxWired_JSpring_08062022.pptx
Wired_JSpring_08062022.pptxSimonedeGijt
 
Nlp tutorial using python nltk (simple examples)
Nlp tutorial using python nltk (simple examples)Nlp tutorial using python nltk (simple examples)
Nlp tutorial using python nltk (simple examples)Mokhtar Ebrahim
 

Semelhante a CSTalks-Natural Language Processing-2 Nov (20)

AI_08_NLP.pptx
AI_08_NLP.pptxAI_08_NLP.pptx
AI_08_NLP.pptx
 
Portuguese Linguistic Tools: What, Why and How
Portuguese Linguistic Tools: What, Why and HowPortuguese Linguistic Tools: What, Why and How
Portuguese Linguistic Tools: What, Why and How
 
CS269-01 (1).pptx
CS269-01 (1).pptxCS269-01 (1).pptx
CS269-01 (1).pptx
 
Natural Language Processing
Natural Language ProcessingNatural Language Processing
Natural Language Processing
 
Testing speaking
Testing speakingTesting speaking
Testing speaking
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language Processing
 
srinu.pptx
srinu.pptxsrinu.pptx
srinu.pptx
 
Natural language processing
Natural language processingNatural language processing
Natural language processing
 
Natural language processing for requirements engineering: ICSE 2021 Technical...
Natural language processing for requirements engineering: ICSE 2021 Technical...Natural language processing for requirements engineering: ICSE 2021 Technical...
Natural language processing for requirements engineering: ICSE 2021 Technical...
 
Transfer_Learning_for_Natural_Language_P_v3_MEAP.pdf
Transfer_Learning_for_Natural_Language_P_v3_MEAP.pdfTransfer_Learning_for_Natural_Language_P_v3_MEAP.pdf
Transfer_Learning_for_Natural_Language_P_v3_MEAP.pdf
 
Pycon India 2018 Natural Language Processing Workshop
Pycon India 2018   Natural Language Processing WorkshopPycon India 2018   Natural Language Processing Workshop
Pycon India 2018 Natural Language Processing Workshop
 
thesis_palogiannidi
thesis_palogiannidithesis_palogiannidi
thesis_palogiannidi
 
13. Constantin Orasan (UoW) Natural Language Processing for Translation
13. Constantin Orasan (UoW) Natural Language Processing for Translation13. Constantin Orasan (UoW) Natural Language Processing for Translation
13. Constantin Orasan (UoW) Natural Language Processing for Translation
 
Nlp final
Nlp finalNlp final
Nlp final
 
ICDM 2019 Tutorial: Speech and Language Processing: New Tools and Applications
ICDM 2019 Tutorial: Speech and Language Processing: New Tools and ApplicationsICDM 2019 Tutorial: Speech and Language Processing: New Tools and Applications
ICDM 2019 Tutorial: Speech and Language Processing: New Tools and Applications
 
Yves Peirsman - Deep Learning for NLP
Yves Peirsman - Deep Learning for NLPYves Peirsman - Deep Learning for NLP
Yves Peirsman - Deep Learning for NLP
 
The Translation Game
The Translation GameThe Translation Game
The Translation Game
 
Eta2006 Nona
Eta2006 NonaEta2006 Nona
Eta2006 Nona
 
Wired_JSpring_08062022.pptx
Wired_JSpring_08062022.pptxWired_JSpring_08062022.pptx
Wired_JSpring_08062022.pptx
 
Nlp tutorial using python nltk (simple examples)
Nlp tutorial using python nltk (simple examples)Nlp tutorial using python nltk (simple examples)
Nlp tutorial using python nltk (simple examples)
 

Mais de cstalks

CSTalks-Sensor-Rich Mobile Video Indexing and Search-17Aug
CSTalks-Sensor-Rich Mobile Video Indexing and Search-17AugCSTalks-Sensor-Rich Mobile Video Indexing and Search-17Aug
CSTalks-Sensor-Rich Mobile Video Indexing and Search-17Augcstalks
 
CSTalks-Visualizing Software Behavior-14Sep
CSTalks-Visualizing Software Behavior-14SepCSTalks-Visualizing Software Behavior-14Sep
CSTalks-Visualizing Software Behavior-14Sepcstalks
 
CSTalks-Polymorphic heterogeneous multicore systems-17Aug
CSTalks-Polymorphic heterogeneous multicore systems-17AugCSTalks-Polymorphic heterogeneous multicore systems-17Aug
CSTalks-Polymorphic heterogeneous multicore systems-17Augcstalks
 
CSTalks-Quaternary Semantics Recomandation System-24 Aug
CSTalks-Quaternary Semantics Recomandation System-24 AugCSTalks-Quaternary Semantics Recomandation System-24 Aug
CSTalks-Quaternary Semantics Recomandation System-24 Augcstalks
 
CSTalks - Object detection and tracking - 25th May
CSTalks - Object detection and tracking - 25th MayCSTalks - Object detection and tracking - 25th May
CSTalks - Object detection and tracking - 25th Maycstalks
 
CSTalks - The Multicore Midlife Crisis - 30 Mar
CSTalks - The Multicore Midlife Crisis - 30 MarCSTalks - The Multicore Midlife Crisis - 30 Mar
CSTalks - The Multicore Midlife Crisis - 30 Marcstalks
 
CSTalks - On machine learning - 2 Mar
CSTalks - On machine learning - 2 MarCSTalks - On machine learning - 2 Mar
CSTalks - On machine learning - 2 Marcstalks
 
CSTalks - Real movie recommendation - 9 Mar
CSTalks - Real movie recommendation - 9 MarCSTalks - Real movie recommendation - 9 Mar
CSTalks - Real movie recommendation - 9 Marcstalks
 
CSTalks-LifeBeyondPhD-16Mar
CSTalks-LifeBeyondPhD-16MarCSTalks-LifeBeyondPhD-16Mar
CSTalks-LifeBeyondPhD-16Marcstalks
 
CSTalks - Music Information Retrieval - 23 Feb
CSTalks - Music Information Retrieval - 23 FebCSTalks - Music Information Retrieval - 23 Feb
CSTalks - Music Information Retrieval - 23 Febcstalks
 

Mais de cstalks (10)

CSTalks-Sensor-Rich Mobile Video Indexing and Search-17Aug
CSTalks-Sensor-Rich Mobile Video Indexing and Search-17AugCSTalks-Sensor-Rich Mobile Video Indexing and Search-17Aug
CSTalks-Sensor-Rich Mobile Video Indexing and Search-17Aug
 
CSTalks-Visualizing Software Behavior-14Sep
CSTalks-Visualizing Software Behavior-14SepCSTalks-Visualizing Software Behavior-14Sep
CSTalks-Visualizing Software Behavior-14Sep
 
CSTalks-Polymorphic heterogeneous multicore systems-17Aug
CSTalks-Polymorphic heterogeneous multicore systems-17AugCSTalks-Polymorphic heterogeneous multicore systems-17Aug
CSTalks-Polymorphic heterogeneous multicore systems-17Aug
 
CSTalks-Quaternary Semantics Recomandation System-24 Aug
CSTalks-Quaternary Semantics Recomandation System-24 AugCSTalks-Quaternary Semantics Recomandation System-24 Aug
CSTalks-Quaternary Semantics Recomandation System-24 Aug
 
CSTalks - Object detection and tracking - 25th May
CSTalks - Object detection and tracking - 25th MayCSTalks - Object detection and tracking - 25th May
CSTalks - Object detection and tracking - 25th May
 
CSTalks - The Multicore Midlife Crisis - 30 Mar
CSTalks - The Multicore Midlife Crisis - 30 MarCSTalks - The Multicore Midlife Crisis - 30 Mar
CSTalks - The Multicore Midlife Crisis - 30 Mar
 
CSTalks - On machine learning - 2 Mar
CSTalks - On machine learning - 2 MarCSTalks - On machine learning - 2 Mar
CSTalks - On machine learning - 2 Mar
 
CSTalks - Real movie recommendation - 9 Mar
CSTalks - Real movie recommendation - 9 MarCSTalks - Real movie recommendation - 9 Mar
CSTalks - Real movie recommendation - 9 Mar
 
CSTalks-LifeBeyondPhD-16Mar
CSTalks-LifeBeyondPhD-16MarCSTalks-LifeBeyondPhD-16Mar
CSTalks-LifeBeyondPhD-16Mar
 
CSTalks - Music Information Retrieval - 23 Feb
CSTalks - Music Information Retrieval - 23 FebCSTalks - Music Information Retrieval - 23 Feb
CSTalks - Music Information Retrieval - 23 Feb
 

Último

Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxKatherine Villaluna
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxDr. Santhosh Kumar. N
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17Celine George
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17Celine George
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxMYDA ANGELICA SUAN
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and stepobaje godwin sunday
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice documentXsasf Sfdfasd
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 

Último (20)

Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptx
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptx
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptx
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and step
 
The Singapore Teaching Practice document
The Singapore Teaching Practice documentThe Singapore Teaching Practice document
The Singapore Teaching Practice document
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 

CSTalks-Natural Language Processing-2 Nov

  • 1. Natural Language Processing Daniel Dahlmeier NUS Graduate School for Integrative Sciences and Engineering danielhe@comp.nus.edu.sg CSTalks 2 November 2011
  • 2. Acknowledgments Examples and figures from Michael Collins’ lecture notes: http://www.cs.columbia.edu/∼mcollins. Some other figures are from Wikipedia: http://www.wikipedia.org. The rest I randomly found on the web.
  • 3. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Google translate 3/25
  • 4. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion IBM’s Watson computer wins at Jeopardy! 4/25
  • 5. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Siri 5/25
  • 6. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion What is Natural Language Processing? Natural Language Processing (NLP) or Computational Linguistics Language processing that goes beyond a “bag of words” representation. Example Translate from one language into the other. Answer natural language questions. Parse the syntactic/semantic structure of a sentence. The other NLP NLP = neuro-linguistic programming. 6/25
  • 7. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Background(s): Artificial Intelligence Talk to your computer Dave: Hello, HAL. Do you read me, HAL? HAL: Affirmative, Dave. I read you. Dave: Open the pod bay doors, HAL. HAL: I’m sorry, Dave. I’m afraid I can’t do that. The computer needs to ... Understand the user : Natural Language Understanding. Generate a well-formed reply : Natural Language Generation. 7/25
  • 8. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Background(s): Artificial Intelligence (cont.) Turing Test Experimenter talks to two parties A and B via a terminal. If C cannot distinguish which party is a computer and which is a human, we should consider the computer to be intelligent. Natural language is deeply intertwined with intelligence. 8/25
  • 9. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Background(s): Linguistics Generative Linguistics Humans can produce and understand an infinite number of sentences by means of a finite set of rules. Language is produced through a generative, recursive process in the human brain. The principles that underlie this process are universal to all languages (universal grammar). 9/25
  • 10. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Background(s): the Web “We are drowning in information but starved for knowledge.” by Edward Osborne Wilson Too much text to read... Wikipedia: over 3.7 million articles (English). PubMed: over 20 million citations. WWW: billions of pages, trillions of words. 10/25
  • 11. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Part-of-speech Tagging Part-of-speech tagging Input: a sentence. Output: a part-of-speech tag sequence, e.g., noun, verb, adjective,... Example Profits/N soared/V at/P Boeing/N Co./N ,/, easily/ADV topping/V forecasts/N on/P Wall/N Street/N ./. 11/25
  • 12. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Named-entity recognition Named-entity recognition Input: a sentence. Output: a BIO-named entity tag sequence, e.g., PERSON, ORGANIZATION, OTHER. Example Profits/O soared/O at/O Boeing/B-ORG Co./I-ORG ,/O easily/O topping/O forecasts/O on/O Wall/O Street/O ./O 12/25
  • 13. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Word Sense Disambiguation Word sense disambiguation Input: a sentence. Output: the sense of each word in the sentence. Example I/sense1 can/sense1 can/sense2 a/sense1 can sense3 . 13/25
  • 14. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Parsing Parsing Input: a sentence. Output: the syntactic tree structure of the sentence. Example Boeing is located in Seattle. 14/25
  • 15. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Machine translation Machine Translation Input: a sentence in language F . Output: the translated sentence in language E . Example Input: Syriens Pr¨sident Baschar al-Assad hat den Westen davor a gewarnt, sich in die Angelegenheiten seines Landes einzumischen. Output: Syrian President Bashar al-Assad has warned the West against interfering in the affairs of his country. 15/25
  • 16. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Why is it hard? ( example from L.Lee) “At last, a computer that understands you like your mother” 16/25
  • 17. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Ambiguity of Natural Language “At last, a computer that understands you like your mother” This could mean... 1 It understands you as well as your mother understands you. 2 It understands (that) you like your mother. 3 It understands you as well as it understands your mother. 1 and 3: Does this mean well, or poorly? 17/25
  • 18. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Ambiguity at the Acoustic Level “At last, a computer that understands you like your mother” This sounds like... 1 “... a computer that understands you like your mother.” 2 “... a computer that understands you lie cured mother.” 18/25
  • 19. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Ambiguity at the Syntactic (structure) Level “At last, a computer that understands you like your mother” 19/25
  • 20. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Ambiguity at the Syntactic (structure) Level “List all flights on Tuesday.” 20/25
  • 21. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Ambiguity at the Semantic (meaning) Level Definition of “mother” 1 a woman who has given birth to a child 2 a stringy slimy substance consisting of yeast cells and bacteria; is added to cider or wine to produce vinegar. More ambiguity They put money in the bank (= buried in mud?). I saw her duck with a telescope (= a duck carrying a telescope?). 21/25
  • 22. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Ambiguity at the Discourse (multi-clause) Level Anaphora resolution Alice says they’ve built a computer that understands you like your mother. But she ... ... doesn’t know any details (Alice) ... doesn’t understand me at all (my mother) 22/25
  • 23. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Related Stuff Machine Learning This really made large-scale, open domain NLP applications possible. Information Retrieval Both need to “understand” language. Linguistics Interested in the nature of language. Psychology / Cognitive Science Both interested in human cognitive capabilities. 23/25
  • 24. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Conclusion What I have told you... What NLP is about. Some NLP tasks that people work on. Why it’s not that easy. What I haven’t told you How do you solve all these problems? How well does it work? What is left to be done? 24/25
  • 25. Examples What is NLP? Background NLP tasks Why is it hard? Related Stuff Conclusion Would you like to know more? NLP courses at NUS CS4248: natural language processing CS6207: advanced natural language processing Books Jurafsky and Martin, Speech and Language Processing (2nd Edition) 25/25