Min-Woo Choi
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
E-mail: choimin1231@catholic.ac.kr
2023. 03. 06 Publish: 2023
Journal: Energy (IF: 11.45)

1
 Introduction
• Background
• Purpose
 Methodology
• Data description
• Experiment setup
 Results
 Conclusions
 Limitation and Future work

2
1. Introduction
Limitation
• Previous studies have focused on either physical models based on detailed atmospheric laws
or statistical approaches such as machine learning algorithms and time series analysis
techniques
• These methods are limited in their ability to capture the complex spatio-temporal correlations of
wind resources which can lead to inaccurate forecasts.
• The purpose of this paper is to present a generic framework for multi-step spatio-temporal
forecasting, using Graph Neural Networks (GNNs) and optional update functions such as
Transformer or LSTM networks.
Purpose of study

3
2. Methodology
Data description
• Location: Norwegian Meteorological Institute
• Period : June 23, 2015, and February 28, 2022
• Number of data: 14 out of 25 available stations
• Resolution: 10 min
• Variable: 8 feature (air temperature, air pressure, dew point,
relative humidity, wind direction, wind speed …)
• Pre-processing: If measurements for a single time-step
were missing for a particular station, linear interpolation
was used to fill the missing entries.
• Training/Validation/Test set: 60%/20%/20%
feature matrix, 𝑓𝑡 ∈ RNx8

4
2. Methodology
Experiment setup
• Look-back window of 32 time-steps was suitable for the 10-min and 1-h ahead forecasts
• Increased to 64 for the 4-h forecasts
Multilevel wavelet decomposition

5
2. Methodology
Fast Fourier Transformer
• FFTtransformer based on signal decomposition and an adapted attention mechanism,
named FFT-Attention.
• FFTransformer is comprised of two streams one which analyses signals with clear
periodicity and another that should learn trend components
• The basic configuration is the same as that of a original transformer, but to better
facilitate the analysis of periodic signals in the left-hand stream, we introduce the FFT-
Attention mechanism.
 FFT is applied to the key, query and value inputs
 Real and imaginary FFT outputs are concatenated with the frequency values, to provide
information on the corresponding frequency for the values in a particular position, similar to
the motivation behind positional encoding, before being fed to an MHA block.
• Outputs from the FFT-Attention module are then concatenated and projected, then
inverse FFT transform values back to the time domain.

6
2. Methodology
Spatial-temporal framework
 All models were constructed in an encoder-only setting, i.e. without
decoders, and used as update functions, 𝜙 (⋅) , in two-layer GNNs.
 Considering a node, 𝑖, its input features were 𝑣𝑖 ∈ R1×(𝑆+𝑃)×8
 𝑆 = historical look-back window
 𝑃 = Predict the next time-steps

7
2. Methodology
Benchmark models
• ST-MLP
• ST-LSTM
• ST-Transformer
• ST-LogSparse
• ST-Informer
• ST-Autoformer
• ST-FFTransformer

8
3. Results
Results of 10 min – 4 h prediction

9
3. Results
Graph connectivity

10
4. Conclusions
• The novel FFTransfomer was proposed which is based on signal decomposition using
wavelet transform and an adapted attention mechanism in the frequency domain - making
it significantly outperforming all other models for 4 hour ahead forecasts.

11
Thank you!