Deep Learning Architectures for NLP (Hungarian NLP Meetup 2016-09-07)
Seu SlideShare está sendo baixado.
×
Confira estes a seguir
Recomendadas
Mais Conteúdo rRelacionado
Diapositivos para si (20)
Semelhante a An LSTM-Based Neural Network Architecture for Model Transformations (20)
Mais de Jordi Cabot (20)
Mais recentes (20)
An LSTM-Based Neural Network Architecture for Model Transformations
- 1. An LSTM-Based Neural Network Architecture for Model Transformations Loli Burgueño, Jordi Cabot, Sébastien Gérard MODELS’19 Munich, September 20th, 2019
- 2. 2
- 3. Artificial Intelligence • Machine Learning - Supervised Learning: 3 Input Output Training Transforming ML Input OutputML Artificial Intelligence Machine Learning Artificial Neural Networks Deep Artificial Neural Networks
- 4. Artificial Neural Networks • Graph structure: Neurons + directed weighted connections • Neurons are mathematical functions • Connections are weights • Adjusted during the learning process to increase/decrease the strength of the connection 4
- 5. Artificial Neural Networks • The learning process basically means to find the right weights • Supervised learning methods. Training phase: • Example input-output pairs are used (Dataset) 5 Dataset Training Validation Test
- 6. Artificial Neural Networks • Combine two LSTM for better results • Avoids fixed size input and output constraints 6 • MTs ≈ sequence-to-sequence arch
- 7. Architecture • Encoder-decoder architecture + • Long short-term memory neural networks 7 Encoder LSTM network Decoder LSTM network InputModel OutputModel
- 8. Architecture • Sequence-to-Sequence transformations • Tree-to-tree transformations • Input layer to embed the input tree to a numeric vector + • Output layer to obtain the output model from the numeric vectors produced by the decoder 8 InputTree EmbeddingLayer Encoder LSTM network OutputTree ExtractionLayer Decoder LSTM network InputModel OutputModel
- 9. • Attention mechanism • To pay more attention (remember better) to specific parts • It automatically detects to which parts are more important 9 Architecture InputTree EmbeddingLayer Encoder LSTM network OutputTree ExtractionLayer Decoder LSTM network AttentionLayer InputModel OutputModel
- 10. • Pre- and post-processing required to… • represent models as trees • reduce the size of the training dataset by using a canonical form • rename variables to avoid the “dictionary problem” 10 Model pre- and post-processing InputModel (preprocessed) InputTree EmbeddingLayer Encoder LSTM network OutputTree ExtractionLayer OutputModel (non-postprocessed) Decoder LSTM network AttentionLayer InputModel OutputModel Preprocessing Postprocessing
- 11. Preliminary results Class to Relational case study 11
- 12. Model representation 12 MODEL ASSOCOBJ c Class ATTS isAbstract name false family OBJ a Attribute ATTS multivalued name false surname OBJ dt Datatype ATTS name String att c a ASSOC type a dt
- 13. Preliminary results Correctness • Measured through the accuracy and validation loss 13
- 14. Preliminary results 14 Performance 1. How long does it take for the training phase to complete?
- 15. Preliminary results Performance 1. How long does it take for the training phase to complete? 15 2. How long it takes to transform an input model when the network is trained?
- 16. Limitations/Discussion • Size of the training dataset • Diversity in the training set • Computational limitations of ANNs • i.e., mathematical operations • Generalization problem • predicting output solutions for input models very different from the training distribution it has learn from • Social acceptance 16
- 17. An LSTM-Based Neural Network Architecture for Model Transformations Loli Burgueño, Jordi Cabot, Sébastien Gérard MODELS’19 Munich, September 20th, 2019
