NS-CUK Seminar: J.H.Lee, Review on "Rethinking the Expressive Power of GNNs via Graph Biconnectivity", ICLR 2023

1. Joo-Ho Lee
School of Computer Science and Information Engineering,
The Catholic University of Korea
E-mail: jooho414@gmail.com
2023-05-05

2. 1
Introduction
Problem Statement
• Most of these works mainly justify their expressiveness by giving toy examples where WL algorithms fail to
distinguish
 On the theoretical side, it is quite unclear what additional power they can systematically and provably gain
• There is still a lack of principled and convincing metrics beyond the WL hierarchy to formally measure the
expressive power and to guide the design of provably better GNN architectures

3. 2
Introduction
Problem Statement
• Biconnectivity provides a structural explanation by breaking down the intrinsic structure of the graph, connecting
it, and making it into a tree structure
• Problems related to biconnectivity can be solved efficiently through classical algorithms, and it is expected that
there will be GNNs that can solve these problems
 However, in this paper, contrary to these expectations, a deep analysis of four representative GNN
structures found that none of them solved the biconnectivity problem

4. 3
Introduction
Contribution
• They systematically study the problem of designing expressive GNNs from a novel perspective of graph
biconnectivity
• They analyze the new GNN structure, Equivariant Subgraph Aggregation Network (ESAN), and demonstrate
that the DSS-WL algorithm can accurately identify cut vertices and cut edges
• Through this, they have expanded understanding of the expressive power of the DSS-WL algorithm and recent
extensions, as well as providing a fine-grained analysis of key factors such as graph generation policies and
aggregation methods
• The main contribution in this paper is then to give a principled and efficient way to design GNNs that are
expressive for biconnectivity problems

5. 4
Methodology
Generalized Distance Weisfeiler-Lehman Test
• Generalized Distance Weisfeiler-Lehman
𝜒𝐺
𝑡
𝑣 ≔ ℎ𝑎𝑠ℎ({ 𝑑𝐺 𝑣, 𝑢 , 𝜒𝐺
𝑡−1
𝑢 : 𝑢 ∈ 𝑉 })

6. 5
Methodology
Generalized Distance Weisfeiler-Lehman Test
• SPD-WL for edge-biconnectivity
 SPD-WL is a more complex algorithm that determines the color of each node by aggregating the colors of all
nodes within the k-distance as well as neighboring nodes

7. 6
Methodology
Generalized Distance Weisfeiler-Lehman Test
• RD-WL for vertex-biconnectivity
 It shows that there is not enough expression for the vertex-biconnectivity problem.
 To overcome this, this paper proposes a new distance measurement method called Resistance Distance (RD)

8. 7
Methodology
Practical implementation : Graphormer-GD
𝐘ℎ
= 𝜙1
ℎ
𝐃 ⨀𝑠𝑜𝑓𝑚𝑎𝑥 𝐗𝐖𝑄
ℎ
𝐗𝐖𝐾
ℎ ⊤
+ 𝜙2
ℎ
𝐷 𝐗𝐖𝑉
ℎ
𝑌 =
ℎ
𝑌ℎ𝑊
𝑜
ℎ

9. 8
Experiment
Baselines
• MPNN
• GIN
• GraphSAGE
• GAT
• GCN
• MoNet
• GatedGCN-PE
• MPNN(sum)
• PNA
• Higher-order GNNs
• RingGNN
• 3WLGNN
• Substructure-based GNNs
• GSN
• CIN-Small
• Subgraph GNNs
• NGNN
• DSS-GNN
• GNN-AK
• GNN-AK+
• SUN
• Graph Transformers
• GT
• SAN
• Graphormer
• URPE

10. 9
Experiment
Accuracy on cut vertex and cut edge detection tasks

11. 10
Experiment
MAE on ZINC test set

12. 11
Conclusion
• In this paper, they systematically investigate the expressive power of GNNs via the perspective of graph
biconnectivity
• They then introduce the principled GD-WL framework that is fully expressive for all biconnectivity metrics
• They further design the Graphormer-GD architecture that is provably powerful while enjoying practical efficiency
and parallelizability
• Experiments on both synthetic and real-world datasets demonstrate the effectiveness of Graphormer-GD

13. 12
Conclusion
1. it remains an important open problem whether biconnectivity can be solved more efficiently in 𝑂(𝑛2) time using
equivariant GNNs
2. a deep understanding of GD-WL is generally lacking.
3. it may be interesting to further investigate more expressive distance (structural) encoding schemes beyond RD-
WL and explore how to encode them in Graph Transformers
4. Finally, one can extend biconnectivity to a hierarchy of higher-order variants (e.g., tri-connectivity), which
provides a completely different view parallel to the WL hierarchy to study the expressive power and guide
designing provably powerful GNNs architectures
There are still many promising directions that have not yet been explored