NLP

1. NATURAL LANGUAGE
PROCESSING Shallote Dsouza

2. WHAT IS NATURAL LANGUAGE
PROCESSING?
Natural Language Processing (NLP) refers to AI method of
communicating with an intelligent system using a natural language
such as English, Spanish, Hindi etc.
The goal of natural language processing is to allow
nonprogrammers to obtain useful information from computing
systems or give commands to the computing system using natural
languages which they may speak or write.
There is a vast store of information recorded in the Natural

3. WHY USE NATURAL LANGUAGE
PROCESSING
 Helps computers communicate with humans in their own language and scales other
language-related tasks
 It helps resolve ambiguity in language and adds useful numeric structure to the data
for many downstream applications, such as speech recognition or text analytics.
 Content categorization: A linguistic-based document summary, including search and
indexing, content alerts and duplication detection.
Topic discovery and modeling: Accurately capture the meaning and themes in text
collections, and apply advanced analytics to text, like optimization and forecasting.

4.  Contextual extraction: Automatically pull structured information from text-
based sources.
 Sentiment analysis: Identifying the mood or subjective opinions within large
amounts of text, including average sentiment and opinion mining.
 Speech-to-text and text-to-speech conversion: Transforming voice
commands into written text, and vice versa.
 Document summarization: Automatically generating synopses of large bodies
of text.

5. COMPONENTS OF NLP
Natural Language Understanding
Mapping the given input in natural language into useful representations i.e.
Taking some spoken/typed sentence and working out what it means
Different level of analysis required:
•Morphological analysis
•Syntactic analysis
•Semantic analysis
•Discourse analysis

6. Natural Language Generation
Producing meaningful phrases and sentences in the form of natural language
from some internal representation i.e. Taking some formal representation of what
you want to say and working out a way to express it in a natural (human)
language (e.g., English)
 Different level of synthesis required:
•Deep planning (what to say)
•Syntactic generation
NL Understanding is much more difficult than NL Generation.

7. STEPS OF NLP

8. MORPHOLOGICAL AND LEXICAL
ANALYSIS
The lexicon of a language is its vocabulary that includes its words
and expressions
Morphology depicts analyzing, identifying and description of
structure of words
Lexical analysis involves dividing a text into paragraphs, words and
the sentences

9. SYNTACTIC ANALYSIS
 Syntax concerns the proper ordering of words and its affect on
meaning
This involves analysis of the words in a sentence to depict the
grammatical structure of the sentence
The words are transformed into structure that shows how the words
are related to each other
 Eg. “the girl the go to the school”. This would definitely be rejected
by the English syntactic analyzer

10. SEMANTIC ANALYSIS
 Semantics concerns the (literal) meaning of words, phrases and
sentences
 This abstracts the dictionary meaning or the exact meaning from
context
 The structures which are created by the syntactic analyzer are
assigned meaning
 E.g.. “colorless blue idea” .This would be rejected by the analyzer as
colorless blue do not make any sense together

11. DISCOURSE INTEGRATION
 Sense of the context
 The meaning of any single sentence depends upon the sentences
that precedes it and also invokes the meaning of the sentences that
follow it
 E.g. the word “it” in the sentence “she wanted it” depends upon the
prior discourse context

12. PRAGMATIC ANALYSIS
 Pragmatics concerns the overall communicative and social context and its
effect on interpretation
 It means abstracting or deriving the purposeful use of the language in
situations
 Importantly those aspects of language which require world knowledge
 The main focus is on what was said is reinterpreted on what it actually means
 E.g. “close the window?” should have been interpreted as a request rather than

13. NATURAL LANGUAGE GENERATION
NLG is the process of constructing natural language outputs from non-linguistic inputs
 NLG can be viewed as the reverse process of NL understanding
 A NLG system may have three main parts:
 Discourse Planner
what will be generated. which sentences
 Surface Realizer
realizes a sentence from its internal representation
 Lexical Selection
selecting the correct words describing the concepts

14. APPLICATION OF NLP
 Search Autocorrect and Autocomplete
 Language Translator
 Social Media Monitoring
 Chatbots
 Survey Analysis
 Targeted Advertising
 Hiring and Recruitment

15. CHALLENGES WITH NLP
Ambiguity
• Lexical ambiguity
- Treating the word “board” as noun or verb?
•Syntactical ambiguity
- “He lifted the beetle with red cap”
- Did he use cap to lift the beetle or he lifted a beetle that had red cap?
•Referential ambiguity
- Rima went to Gauri. She said, “I am tired.”
- Exactly who is tired?

16. Phrases / Idioms
“A perfect storm” means The worst possible situation
 Connecting language and machine perception
 Sentence generation
 Text summarization
 Keyword extraction

17. FUTURE OF NLP
 Human level or human readable natural language processing is an AI-complete
problem
 It is equivalent to solving the central artificial intelligence problem and making
computers as intelligent as people
 Make computers as they can solve problems like humans and think like humans
as well as perform activities that humans cant perform and making it more
efficient than humans
 NLP's future is closely linked to the growth of Artificial intelligence
 As natural language understanding or readability improves, computers or
machines or devices will be able to learn from the information online and apply
what they learned in the real world

18. THANK YOU