Denunciar

DongDongChen9Seguir

25 de Mar de 2023•0 gostou## 0 gostaram

•4 visualizações## visualizações

Seja o primeiro a gostar disto

mostrar mais

Vistos totais

0

No Slideshare

0

De incorporações

0

Número de incorporações

0

25 de Mar de 2023•0 gostou## 0 gostaram

•4 visualizações## visualizações

Seja o primeiro a gostar disto

mostrar mais

Vistos totais

0

No Slideshare

0

De incorporações

0

Número de incorporações

0

Baixar para ler offline

Denunciar

Engenharia

Graph neural networks meet reinforcement learning

DongDongChen9Seguir

Graph neural networks overviewRodion Kiryukhin

Colloquium.pptxMythili680896

Node classification with graph neural network based centrality measures and f...IJECEIAES

JOSA TechTalks - Machine Learning on Graph-Structured DataJordan Open Source Association

Visual Search and Question Answering IIWanjin Yu

GraphSignalProcessingFinalPaperChiraz Nafouki

- Graph Neural Networks (GNNs) Meet Reinforcement Learning (RL) 05/01/2021
- Graph Neural Networks (GNNs) Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016). ● Targeting at real-world problems having underlying graph structures which are non-Euclidean. ○ Euclidean data is data which is sensibly modelled as being plotted in n-dimensional linear space, e.g., images (where the x and y coordinates refer to the location of each pixel, and the z coordinate refers to its colour/intensity). ○ Non-Euclidean data: have information of edges, values associated with nodes, or the directionality of edges.
- Graph Neural Networks (GNNs) Tasks
- Graph Neural Networks (GNNs) Neural Networks Forward Propagation ● ● w is weights, H is the last hidden states, b is the bias and sigma is the nonlinear function. ● When i = 0, then H[i] = X, where X is the input. Graph Neural Networks ● Sparse. ● Flexible connection.
- Graph Neural Networks (GNNs) Definitions ● Adjacency matrix (A): a N x N matrix filled with either 0 or 1, where N is the total number of nodes. ● Node attributes matrix (X): represents the features or attributes of each node. If there are N nodes and the size of node attributes is F, then the shape of this matrix is N x F. ● Edge attributes matrix (E), can also have its own attributes. Adjacency matrix Node attributes matrix (6 x 6)
- Graph Neural Networks (GNNs) Graph ● G = (V, E), denote a graph. ● node feature vectors Xv for v in V Graph Neural Networks[1] ● Step1: aggregate information from neighbors. ● Step2: combine encoded neighbor’s info with its own info. ● Step3: use READOUT function to extract the representation.
- Graph Convolutional Neural Network (GCN) Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016). Aggregator + Combine: Representation from the paper ●
- GraphSAGE Hamilton, William L., Rex Ying, and Jure Leskovec. "Inductive representation learning on large graphs." arXiv preprint arXiv:1706.02216 (2017). ● Aggregator: ● Combine: Representation from the paper ● Different aggregator ○ Mean, LSTM, Pooling.
- GraphSAGE Hamilton, William L., Rex Ying, and Jure Leskovec. "Inductive representation learning on large graphs." arXiv preprint arXiv:1706.02216 (2017).
- Literature Review Dong, Jiqian, et al. "A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network." arXiv preprint arXiv:2010.05437 (2020). ● Combining graphic convolution neural network (GCN) and Deep Q Network (DQN) to solve the lane changing problem in mixed traffic with multiple AVs. ● Off-policy Q learning.
- Attention (Self-attention) ● Self-attention, also known as intra-attention, is an attention mechanism relating different positions of a single sequence in order to compute a representation of the same sequence.
- Attention (self-attention, intra-attention) ● Different attention mechanisms.
- Attention (self-attention, intra-attention) ● Born for Translation
- Attention (self-attention, intra-attention) ●
- Attention (self-attention, intra-attention) ●
- Attention (self-attention, intra-attention) ●
- Attention (self-attention, intra-attention) ●
- Attention (self-attention, intra-attention) ●
- Attention (self-attention, intra-attention) ● Learnable attention matrix: WQ, WK and WV.
- Previous Works ● Iqbal, Shariq, and Fei Sha. "Actor-attention-critic for multi-agent reinforcement learning." International Conference on Machine Learning. PMLR, 2019. Our attention critic is able to dynamically select which agents to attend to at each time point during training, improving performance in multi-agent domains with complex interactions.
- Graph Convolutional Reinforcement Learning (DGN) Jiang, Jiechuan, et al. "Graph convolutional reinforcement learning." arXiv preprint arXiv:1810.09202 (2018). ● Graph convolution adapts to the dynamics of the underlying graph of the multi-agent environment, and relation kernels capture the interplay between agents by their relation representations. ● Use self-attention to extract features. ● Off-policy Q learning.
- GAT on Autonomous Driving Wang, Jiawei, et al. "Multi-agent Graph Reinforcement Learning for Connected Automated Driving", 2020 ● On-policy PPO. ● Lane-following.
- References 1. Dong, Jiqian, et al. "A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network." arXiv preprint arXiv:2010.05437 (2020). 2. Gupta, Jayesh K., Maxim Egorov, and Mykel Kochenderfer. "Cooperative multi-agent control using deep reinforcement learning." International Conference on Autonomous Agents and Multiagent Systems. Springer, Cham, 2017. 3. Lin, Kaixiang, et al. "Efficient large-scale fleet management via multi-agent deep reinforcement learning." Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018. 4. https://ai.stackexchange.com/questions/11226/what-is-non-euclidean-data#:~:text=Since%20Euclidean %20spaces%20are%20prototypically,coordinate%20refers%20to%20its%20colour%2F 5. https://towardsdatascience.com/understanding-graph-convolutional-networks-for-node-classification-a 2bfdb7aba7b 6. Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016).
- Thanks!