SlideShare uma empresa Scribd logo
1 de 1
Baixar para ler offline
Sequential Labeling for Tracking Dynamic Dialog States
Seokhwan Kim, Rafael E. Banchs
Natural Language Understanding Lab., Human Language Technology Department,
Institute for Infocomm Research (I2
R), Singapore
Introduction
Dialog state tracking is a sub-task of dialog management that
analyzes and maintains the dialog state at each moment
The major obstacle to dialog state tracking is the errors
produced by ASR and SLU
This work focuses on the second dialog state tracking
challenge (DSTC 2) with three subtasks:
Goals Tracking
Method Tracking
Requested Slots Tracking
Problem Definition: Goals Tracking
Goals represent the constraint values which are truly intended
by a user at each moment
A problem of finding the distributions over the hypotheses for
the following four categories:
AREA
FOOD
NAME
PRICE RANGE
The models for DSTC 2 should be able to handle goal changes
during a session
An example of goal change in DSTC 2
Utterance
Goals
Food Area
S1 Hello, How may I help you?
U1 I need a Persian restaurant in the south part
of town.
Persian South
S2 What kind of food would you like?
U2 Persian. Persian South
S3 I’m sorry but there is no restaurant serving
persian food
U3 How about Portuguese food? Portuguese South
S4 Peking restaurant is a nice place in the south
of town.
U4 Is that Portuguese? Portuguese South
S5 Nandos is a nice place in the south of town
serving tasty Portuguese food.
U5 Alright. Whats the phone number? Portuguese South
S6 The phone number of nandos is 01223
327908 .
U6 And the address? Portuguese South
S7 Sure, nandos is on Cambridge Leisure Park
Clifton Way.
U7 Thank you good bye.
Problem Definition: Method Tracking
To classify the way of requesting information by a user into the
following four categories: ‘by constraints’, ‘by alternatives’, ‘by
name’, ‘finished’
The probability distribution is computed for each turn
Problem Definition: Requested Slots Tracking
To specify the slots requested by a user
The binary distributions are computed with the probabilities
whether each slot is requested or not
Requestable slots: AREA, FOOD, NAME, PRICERANGE,
ADDR, PHONE, POSTCODE, SIGNATURE
Method: Sequential Labeling of Dialog States
To produce the most probable label sequence y = {y1, · · · , yn}
of a given input sequence x = {x1, · · · , xn}
BIO tagging scheme
To detect the boundaries of the label chunks
Considering discourse coherences in conversation
An example of goal chain on the food slot
Linear Chain CRFs
Conditional probability distributions over the label sequences y
conditioned on the input sequence x
p (y|x) =
1
Z (x)
n
t=1
Ψ(yt, yt−1, x),
Ψ(yt, yt−1, x) = Ψ1(yt, x) · Ψ2(yt, yt−1)
Ψ1(yt, x) = exp ( k λkfk(yt, x))
Ψ2(yt, yt−1) = exp ( k λkfk(yt, yt−1))
Experimental Settings
DSTC 2 dataset
3,235 dialog sessions on restaurant information domain
TRAINING: 1,612 sessions
DEVELOPMENT: 506 sessions
TEST: 1,117 sessions
The results of ASR and SLU are annotated for every turn in the dataset,
as well as the gold standard annotations are also provided for evaluation
Models
CRF (Conditional Random Fields): with sequential labeling
ME (Maximum Entropy): without sequential labeling
Features
SLU Hypothesis: inform, confirm, deny, affirm, negate, request, reqalts
System Action: expl-conf, impl-conf, request, select, canthelp
Evaluation Metrics
Features metrics: Accuracy, L2 norm, ROC CA 5
On Joint Goals, Method, and Requested Slots
Experimental Results
Comparisons of dialog state tracking performances
Dev set Test set
Acc L2 ROC Acc L2 ROC
Joint Goals
ME 0.638 0.551 0.144 0.596 0.671 0.036
CRF 0.644 0.545 0.103 0.601 0.649 0.064
Method
ME 0.839 0.260 0.398 0.877 0.204 0.397
CRF 0.875 0.202 0.181 0.904 0.155 0.187
Requested Slots
ME 0.946 0.099 0.000 0.957 0.081 0.000
CRF 0.942 0.107 0.000 0.960 0.073 0.000
CRF models produced better joint goals and method in
accuracy and L2 norm on both development and test sets
For the requested slots task, our proposed approach achieved
better results than the baseline on the test set
Conclusion
This paper presented a sequential labeling approach for dialog
state tracking
Experimental results show the merits of our proposed approach
with the improved performances on all the sub-tasks of DSTC 2
If we discover more advanced features that help to track the
proper dialog states, they can raise the overall performances
further
1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore 138632 Email: kims@i2r.a-star.edu.sg WWW: http://hlt.i2r.a-star.edu.sg/

Mais conteúdo relacionado

Semelhante a Sequential Labeling for Tracking Dynamic Dialog States

Interpretability and Reproducibility in Production Machine Learning Applicat...
 Interpretability and Reproducibility in Production Machine Learning Applicat... Interpretability and Reproducibility in Production Machine Learning Applicat...
Interpretability and Reproducibility in Production Machine Learning Applicat...Swaminathan Sundararaman
 
"Test Design Techniques"
"Test Design Techniques" "Test Design Techniques"
"Test Design Techniques" HYS Enterprise
 
Towards better software quality assurance by providing intelligent support
Towards better software quality assurance by providing intelligent supportTowards better software quality assurance by providing intelligent support
Towards better software quality assurance by providing intelligent supportConcordia University
 
HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...
HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...
HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...Maribel Acosta Deibe
 
Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...
Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...
Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...Jinho Choi
 
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCLOCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCLLionel Briand
 
Rethinking action spaces for reinforcement learning in end-to-end dialog agen...
Rethinking action spaces for reinforcement learning in end-to-end dialog agen...Rethinking action spaces for reinforcement learning in end-to-end dialog agen...
Rethinking action spaces for reinforcement learning in end-to-end dialog agen...Jeong-Gwan Lee
 
Thesis defense 2017
Thesis defense 2017Thesis defense 2017
Thesis defense 2017yashiitr
 
Isolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantizationIsolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantizationeSAT Journals
 
Isolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantizationIsolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantizationeSAT Publishing House
 
Studies On Traffic Management Models for Wireless Communication Network
Studies On Traffic Management Models for Wireless Communication NetworkStudies On Traffic Management Models for Wireless Communication Network
Studies On Traffic Management Models for Wireless Communication NetworkNeetaSingh38
 
Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...
Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...
Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...paupo
 
Gas station competition_model
Gas station competition_modelGas station competition_model
Gas station competition_modelAri Pramono
 
Simple vs. Compound Mark Hierarchical Marking Menus
Simple vs. Compound Mark Hierarchical Marking MenusSimple vs. Compound Mark Hierarchical Marking Menus
Simple vs. Compound Mark Hierarchical Marking MenusKimberly Aguada
 
Face recognition v1
Face recognition v1Face recognition v1
Face recognition v1San Kim
 
A tale of experiments on bug prediction
A tale of experiments on bug predictionA tale of experiments on bug prediction
A tale of experiments on bug predictionMartin Pinzger
 

Semelhante a Sequential Labeling for Tracking Dynamic Dialog States (20)

Interpretability and Reproducibility in Production Machine Learning Applicat...
 Interpretability and Reproducibility in Production Machine Learning Applicat... Interpretability and Reproducibility in Production Machine Learning Applicat...
Interpretability and Reproducibility in Production Machine Learning Applicat...
 
"Test Design Techniques"
"Test Design Techniques" "Test Design Techniques"
"Test Design Techniques"
 
Ngs part iii 2013
Ngs part iii 2013Ngs part iii 2013
Ngs part iii 2013
 
Towards better software quality assurance by providing intelligent support
Towards better software quality assurance by providing intelligent supportTowards better software quality assurance by providing intelligent support
Towards better software quality assurance by providing intelligent support
 
HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...
HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...
HARE: An Engine for Enhancing Answer Completeness of SPARQL Queries via Crowd...
 
Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...
Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...
Noise Pollution in Hospital Readmission Prediction: Long Document Classificat...
 
Chi Square & Anova
Chi Square & AnovaChi Square & Anova
Chi Square & Anova
 
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCLOCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
OCLR: A More Expressive, Pattern-Based Temporal Extension of OCL
 
Rethinking action spaces for reinforcement learning in end-to-end dialog agen...
Rethinking action spaces for reinforcement learning in end-to-end dialog agen...Rethinking action spaces for reinforcement learning in end-to-end dialog agen...
Rethinking action spaces for reinforcement learning in end-to-end dialog agen...
 
Thesis defense 2017
Thesis defense 2017Thesis defense 2017
Thesis defense 2017
 
Isolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantizationIsolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantization
 
Isolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantizationIsolated word recognition using lpc & vector quantization
Isolated word recognition using lpc & vector quantization
 
Studies On Traffic Management Models for Wireless Communication Network
Studies On Traffic Management Models for Wireless Communication NetworkStudies On Traffic Management Models for Wireless Communication Network
Studies On Traffic Management Models for Wireless Communication Network
 
Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...
Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...
Holistic Analysis and Optimization of Heterogeneous Fault-Tolerant Embedded S...
 
50320130403005 2
50320130403005 250320130403005 2
50320130403005 2
 
Gas station competition_model
Gas station competition_modelGas station competition_model
Gas station competition_model
 
CORRECT-ICSE2016
CORRECT-ICSE2016CORRECT-ICSE2016
CORRECT-ICSE2016
 
Simple vs. Compound Mark Hierarchical Marking Menus
Simple vs. Compound Mark Hierarchical Marking MenusSimple vs. Compound Mark Hierarchical Marking Menus
Simple vs. Compound Mark Hierarchical Marking Menus
 
Face recognition v1
Face recognition v1Face recognition v1
Face recognition v1
 
A tale of experiments on bug prediction
A tale of experiments on bug predictionA tale of experiments on bug prediction
A tale of experiments on bug prediction
 

Mais de Seokhwan Kim

The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)Seokhwan Kim
 
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Seokhwan Kim
 
Dynamic Memory Networks for Dialogue Topic Tracking
Dynamic Memory Networks for Dialogue Topic TrackingDynamic Memory Networks for Dialogue Topic Tracking
Dynamic Memory Networks for Dialogue Topic TrackingSeokhwan Kim
 
The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)Seokhwan Kim
 
Natural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionNatural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionSeokhwan Kim
 
Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...
Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...
Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...Seokhwan Kim
 
The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)Seokhwan Kim
 
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Seokhwan Kim
 
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Seokhwan Kim
 
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...Seokhwan Kim
 
Wikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingWikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingSeokhwan Kim
 
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...Seokhwan Kim
 
MMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionMMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionSeokhwan Kim
 
A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...Seokhwan Kim
 
A spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessA spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessSeokhwan Kim
 
An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...Seokhwan Kim
 
An Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionAn Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionSeokhwan Kim
 
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템Seokhwan Kim
 
A Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionA Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionSeokhwan Kim
 
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...Seokhwan Kim
 

Mais de Seokhwan Kim (20)

The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)
 
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
 
Dynamic Memory Networks for Dialogue Topic Tracking
Dynamic Memory Networks for Dialogue Topic TrackingDynamic Memory Networks for Dialogue Topic Tracking
Dynamic Memory Networks for Dialogue Topic Tracking
 
The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)
 
Natural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionNatural Language in Human-Robot Interaction
Natural Language in Human-Robot Interaction
 
Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...
Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...
Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling...
 
The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)
 
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
 
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
 
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
 
Wikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingWikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic Tracking
 
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
 
MMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionMMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognition
 
A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...
 
A spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessA spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information access
 
An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...
 
An Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionAn Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information Extraction
 
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
 
A Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionA Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation Detection
 
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
 

Último

Main Exam Applied biochemistry final year
Main Exam Applied biochemistry final yearMain Exam Applied biochemistry final year
Main Exam Applied biochemistry final yearmarwaahmad357
 
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdfSUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdfsantiagojoderickdoma
 
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...Sérgio Sacani
 
Pests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPRPests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPRPirithiRaju
 
TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)chatterjeesoumili50
 
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrashi Coaching
 
Substances in Common Use for Shahu College Screening Test
Substances in Common Use for Shahu College Screening TestSubstances in Common Use for Shahu College Screening Test
Substances in Common Use for Shahu College Screening TestAkashDTejwani
 
Role of herbs in hair care Amla and heena.pptx
Role of herbs in hair care  Amla and  heena.pptxRole of herbs in hair care  Amla and  heena.pptx
Role of herbs in hair care Amla and heena.pptxVaishnaviAware
 
RCPE terms and cycles scenarios as of March 2024
RCPE terms and cycles scenarios as of March 2024RCPE terms and cycles scenarios as of March 2024
RCPE terms and cycles scenarios as of March 2024suelcarter1
 
Exploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & ResearchExploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & ResearchPrachya Adhyayan
 
Controlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform EnvironmentControlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform EnvironmentRahulVishwakarma71547
 
Alternative system of medicine herbal drug technology syllabus
Alternative system of medicine herbal drug technology syllabusAlternative system of medicine herbal drug technology syllabus
Alternative system of medicine herbal drug technology syllabusPradnya Wadekar
 
Gene transfer in plants agrobacterium.pdf
Gene transfer in plants agrobacterium.pdfGene transfer in plants agrobacterium.pdf
Gene transfer in plants agrobacterium.pdfNetHelix
 
Pests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPRPests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPRPirithiRaju
 
World Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlabWorld Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlabkiyorndlab
 
Principles & Formulation of Hair Care Products
Principles & Formulation of Hair Care  ProductsPrinciples & Formulation of Hair Care  Products
Principles & Formulation of Hair Care Productspurwaborkar@gmail.com
 
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptxSCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptxROVELYNEDELUNA3
 
IB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptxIB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptxUalikhanKalkhojayev1
 

Último (20)

Main Exam Applied biochemistry final year
Main Exam Applied biochemistry final yearMain Exam Applied biochemistry final year
Main Exam Applied biochemistry final year
 
Applying Cheminformatics to Develop a Structure Searchable Database of Analyt...
Applying Cheminformatics to Develop a Structure Searchable Database of Analyt...Applying Cheminformatics to Develop a Structure Searchable Database of Analyt...
Applying Cheminformatics to Develop a Structure Searchable Database of Analyt...
 
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdfSUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
SUKDANAN DIAGNOSTIC TEST IN PHYSICAL SCIENCE ANSWER KEYY.pdf
 
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
 
Pests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPRPests of ragi_Identification, Binomics_Dr.UPR
Pests of ragi_Identification, Binomics_Dr.UPR
 
TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)TORSION IN GASTROPODS- Anatomical event (Zoology)
TORSION IN GASTROPODS- Anatomical event (Zoology)
 
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्रKrishi Vigyan Kendras - कृषि विज्ञान केंद्र
Krishi Vigyan Kendras - कृषि विज्ञान केंद्र
 
Substances in Common Use for Shahu College Screening Test
Substances in Common Use for Shahu College Screening TestSubstances in Common Use for Shahu College Screening Test
Substances in Common Use for Shahu College Screening Test
 
Role of herbs in hair care Amla and heena.pptx
Role of herbs in hair care  Amla and  heena.pptxRole of herbs in hair care  Amla and  heena.pptx
Role of herbs in hair care Amla and heena.pptx
 
RCPE terms and cycles scenarios as of March 2024
RCPE terms and cycles scenarios as of March 2024RCPE terms and cycles scenarios as of March 2024
RCPE terms and cycles scenarios as of March 2024
 
Exploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & ResearchExploration Method’s in Archaeological Studies & Research
Exploration Method’s in Archaeological Studies & Research
 
Controlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform EnvironmentControlling Parameters of Carbonate platform Environment
Controlling Parameters of Carbonate platform Environment
 
Alternative system of medicine herbal drug technology syllabus
Alternative system of medicine herbal drug technology syllabusAlternative system of medicine herbal drug technology syllabus
Alternative system of medicine herbal drug technology syllabus
 
Gene transfer in plants agrobacterium.pdf
Gene transfer in plants agrobacterium.pdfGene transfer in plants agrobacterium.pdf
Gene transfer in plants agrobacterium.pdf
 
Pests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPRPests of Redgram_Identification, Binomics_Dr.UPR
Pests of Redgram_Identification, Binomics_Dr.UPR
 
World Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlabWorld Water Day 22 March 2024 - kiyorndlab
World Water Day 22 March 2024 - kiyorndlab
 
Principles & Formulation of Hair Care Products
Principles & Formulation of Hair Care  ProductsPrinciples & Formulation of Hair Care  Products
Principles & Formulation of Hair Care Products
 
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptxSCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
SCIENCE 6 QUARTER 3 REVIEWER(FRICTION, GRAVITY, ENERGY AND SPEED).pptx
 
Data delivery from the US-EPA Center for Computational Toxicology and Exposur...
Data delivery from the US-EPA Center for Computational Toxicology and Exposur...Data delivery from the US-EPA Center for Computational Toxicology and Exposur...
Data delivery from the US-EPA Center for Computational Toxicology and Exposur...
 
IB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptxIB Biology New syllabus B3.2 Transport.pptx
IB Biology New syllabus B3.2 Transport.pptx
 

Sequential Labeling for Tracking Dynamic Dialog States

  • 1. Sequential Labeling for Tracking Dynamic Dialog States Seokhwan Kim, Rafael E. Banchs Natural Language Understanding Lab., Human Language Technology Department, Institute for Infocomm Research (I2 R), Singapore Introduction Dialog state tracking is a sub-task of dialog management that analyzes and maintains the dialog state at each moment The major obstacle to dialog state tracking is the errors produced by ASR and SLU This work focuses on the second dialog state tracking challenge (DSTC 2) with three subtasks: Goals Tracking Method Tracking Requested Slots Tracking Problem Definition: Goals Tracking Goals represent the constraint values which are truly intended by a user at each moment A problem of finding the distributions over the hypotheses for the following four categories: AREA FOOD NAME PRICE RANGE The models for DSTC 2 should be able to handle goal changes during a session An example of goal change in DSTC 2 Utterance Goals Food Area S1 Hello, How may I help you? U1 I need a Persian restaurant in the south part of town. Persian South S2 What kind of food would you like? U2 Persian. Persian South S3 I’m sorry but there is no restaurant serving persian food U3 How about Portuguese food? Portuguese South S4 Peking restaurant is a nice place in the south of town. U4 Is that Portuguese? Portuguese South S5 Nandos is a nice place in the south of town serving tasty Portuguese food. U5 Alright. Whats the phone number? Portuguese South S6 The phone number of nandos is 01223 327908 . U6 And the address? Portuguese South S7 Sure, nandos is on Cambridge Leisure Park Clifton Way. U7 Thank you good bye. Problem Definition: Method Tracking To classify the way of requesting information by a user into the following four categories: ‘by constraints’, ‘by alternatives’, ‘by name’, ‘finished’ The probability distribution is computed for each turn Problem Definition: Requested Slots Tracking To specify the slots requested by a user The binary distributions are computed with the probabilities whether each slot is requested or not Requestable slots: AREA, FOOD, NAME, PRICERANGE, ADDR, PHONE, POSTCODE, SIGNATURE Method: Sequential Labeling of Dialog States To produce the most probable label sequence y = {y1, · · · , yn} of a given input sequence x = {x1, · · · , xn} BIO tagging scheme To detect the boundaries of the label chunks Considering discourse coherences in conversation An example of goal chain on the food slot Linear Chain CRFs Conditional probability distributions over the label sequences y conditioned on the input sequence x p (y|x) = 1 Z (x) n t=1 Ψ(yt, yt−1, x), Ψ(yt, yt−1, x) = Ψ1(yt, x) · Ψ2(yt, yt−1) Ψ1(yt, x) = exp ( k λkfk(yt, x)) Ψ2(yt, yt−1) = exp ( k λkfk(yt, yt−1)) Experimental Settings DSTC 2 dataset 3,235 dialog sessions on restaurant information domain TRAINING: 1,612 sessions DEVELOPMENT: 506 sessions TEST: 1,117 sessions The results of ASR and SLU are annotated for every turn in the dataset, as well as the gold standard annotations are also provided for evaluation Models CRF (Conditional Random Fields): with sequential labeling ME (Maximum Entropy): without sequential labeling Features SLU Hypothesis: inform, confirm, deny, affirm, negate, request, reqalts System Action: expl-conf, impl-conf, request, select, canthelp Evaluation Metrics Features metrics: Accuracy, L2 norm, ROC CA 5 On Joint Goals, Method, and Requested Slots Experimental Results Comparisons of dialog state tracking performances Dev set Test set Acc L2 ROC Acc L2 ROC Joint Goals ME 0.638 0.551 0.144 0.596 0.671 0.036 CRF 0.644 0.545 0.103 0.601 0.649 0.064 Method ME 0.839 0.260 0.398 0.877 0.204 0.397 CRF 0.875 0.202 0.181 0.904 0.155 0.187 Requested Slots ME 0.946 0.099 0.000 0.957 0.081 0.000 CRF 0.942 0.107 0.000 0.960 0.073 0.000 CRF models produced better joint goals and method in accuracy and L2 norm on both development and test sets For the requested slots task, our proposed approach achieved better results than the baseline on the test set Conclusion This paper presented a sequential labeling approach for dialog state tracking Experimental results show the merits of our proposed approach with the improved performances on all the sub-tasks of DSTC 2 If we discover more advanced features that help to track the proper dialog states, they can raise the overall performances further 1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore 138632 Email: kims@i2r.a-star.edu.sg WWW: http://hlt.i2r.a-star.edu.sg/