Presentation @ ISF 2012 - Boston, USA, Hybrid NN-NF model to forecast the electricity spot price in Brazil

1. A Hybrid Neuro-Fuzzy System and
Neuro-
Neural Network Approach to
Forecast the Electricity Spot Price
in Brazil
Mônica Barros, D.Sc.
Lucio de Medeiros, D.Sc.
June, 2012
monica@mbarros.com 1

2. Acknowledgment
This work was sponsored by Norte
Fluminense Thermal Plant (UTE
Norte Fluminense) through a R&D
grant.
monica@mbarros.com 2

3. Contents
Background
Neural networks and Neuro-Fuzzy systems
An Overview of the Data
The Model - Hybrid neural network/neuro-
fuzzy model for the spot price
Comments on the choice of the ANN Model
Comments on the NFS set-up
The structure of the hybrid model
Empirical results
Conclusions
monica@mbarros.com 3

4. Background
The aim is to present a Hybrid Neuro-
Fuzzy/Neural Network model which incorporates
inflow information to forecast the weekly spot
prices in the Southeast subsystem of Brazil.
The Southeast subsystem corresponds to the
most densely populated and industrialized
portion of Brazil.
Part of the region is subject to occasional severe
droughts that impact electricity generation.
monica@mbarros.com 4

5. Background
Power generation in Brazil is primarily hydroelectric,
and hydro plants account for about 82% of the
electricity generation in the country.
Power plants are connected through long distances
by a complex array of power lines, in what forms the
so-called Brazilian interconnected system (SIN).
SIN comprises about 97% of the total energy
produced in the country.
The concept of subsystem is intrinsically related to
the concept of “equivalent reservoir”.
monica@mbarros.com 5

6. Background
Spot prices in Brazil are computed through an
optimization process that attempts to minimize costs
in equivalent reservoirs, one for each of 4
subsystems.
Electricity spot prices are calculated through a
sequence of complex optimization models that
produce the marginal cost of operation.
These models attempt to minimize the total cost of
operations, computed as the sum of current and
future costs.
These costs are functions of (among other variables)
the expected future inflows, the expected demand
and the current reservoir levels.
monica@mbarros.com 6

7. Background
“ONS”, the Brazilian Independent System Operator
(ISO) employs a model based on Stochastic Dual
Dynamic Programming to perform operations
planning.
This model groups hydroelectric power plants of
basins with similar hydrological behavior into the
so-called equivalent subsystems.
Four equivalent subsystems (North, Northeast,
Southeast/Central-West and South regions) are
used.
Hydrological scenarios are built into the system
through a PAR(p) – Periodic Autoregressive Model.
monica@mbarros.com 7

8. Background
Two quantities play an important role to
determine spot prices:
“Stored energy” - maximum storage of the reservoir or
basin;
“Natural inflow energy” – river inflows, expressed in
energy units.
Paranaíba and Grande river basins are the main
basins in the Southeast subsystem, accounting
for slightly over 60% of the subsystem’s
reservoir’s capacity.
monica@mbarros.com 8

9. Neural networks
ANNs have been extensively used in time series
forecasting, due to their generalization and
learning abilities.
They can identify nonlinear characteristics of
complex series.
The architecture of a Mutilayer Perceptron (MLP)
network is:
monica@mbarros.com 9

10. Neuro-
Neuro-Fuzzy systems
Neuro-fuzzy systems attempt to combine the
advantages of both approaches: neural networks
and fuzzy systems.
We use the ANFIS neural fuzzy inference system
proposed by Jang.
monica@mbarros.com 10

11. Spot price (R$/MWh)
0
100
200
300
400
500
05/11/02-05/17/02 600
09/28/02-10/04/02
02/15/03-02/21/03
07/05/03-07/11/03
11/22/03-11/28/03
04/10/04-04/16/04
08/28/04-09/03/04
01/15/05-01/21/05
06/04/05-06/10/05
10/22/05-10/28/05
03/11/06-03/17/06
(May 2002 to May 2011)
07/29/06-08/04/06
12/16/06-12/22/06
week
An Overview of the Data
05/05/07-05/11/07
monica@mbarros.com
09/22/07-09/28/07
02/09/08-02/15/08
Spot Price - Southeast Subsystem
06/28/08-07/04/08
11/15/08-11/21/08
04/04/09-04/10/09
08/22/09-08/28/09
12/09/10-01/15/10
05/29/10-06/04/10
10/16/10-10/22/11
03/05/11-03/11/11
11
The spot series consists of 471 weekly observations

12. An Overview of the Data
From the previous figure two striking features
emerge:
prices tend to stay at very low levels for long periods, but
they also exhibit high volatility.
Both features are common in primarily based
hydroelectric systems, such as the Brazilian.
The high volatility of the series is also due to the
non-storability of electricity and it is observed even
in markets where prices are “actual” market prices,
a consequence of bid and ask interactions, and not
the result of optimization models, as in the Brazilian
case.
monica@mbarros.com 12

13. (%)
-75%
-50%
-25%
0%
100%
125%
150%
-200%
-175%
-150%
-125%
-100%
25%
50%
75%
05/11/02-05/17/02
08/31/02-09/06/02
12/21/02-12/27/02
04/12/03-04/18/03
08/02/03-08/08/03
11/22/03-11/28/03
03/13/04-03/19/04
07/03/04-07/09/04
10/23/04-10/29/04
02/12/05-02/18/05
06/04/05-06/10/05
09/24/05-09/30/05
01/14/06-01/20/06
05/06/06-05/12/06
08/26/06-09/01/06
12/16/06-12/22/06
week
04/07/07-04/13/07
Weekly log Return
07/28/07-08/03/07
11/17/07-11/23/07
03/08/08-03/14/08
06/28/08-07/04/08
10/18/08-10/24/08
02/07/09-02/13/09
05/30/09-06/05/09
09/19/09-09/25/09
12/09/10-01/15/10
05/01/10-05/07/10
08/21/10-08/27/10
12/11/10-12/17/10
04/02/11-04/08/11
Mode
Mean
Count
An Overview of the Data
Median
Kurtosis
Statistics
Minimum
not uncommon in the sample
Maximum
Skewness
monica@mbarros.com
Standard Deviation
Log-returns based on the weekly prices
4.0
471
3.92
23.89
66.54
18.59
27.95
54.33
569.59
Spot Price (R$/MWh)
Extreme weekly returns (in excess of ±50%) are
13
470
7.63
0.0%
0.0%
-0.44
-0.1%
30.6%
138.8%
-194.2%
Weekly Return (%)

14. Natural Inflow Energy (Average MW)
0
10000
15000
20000
25000
5000
05/11/02-05/17/02
08/31/02-09/06/02
12/21/02-12/27/02
04/12/03-04/18/03
08/02/03-08/08/03
11/22/03-11/28/03
03/13/04-03/19/04
07/03/04-07/09/04
10/23/04-10/29/04
02/12/05-02/18/05
06/04/05-06/10/05
09/24/05-09/30/05
01/14/06-01/20/06
Natural Inflow Energy (Paranaíba)
05/06/06-05/12/06
08/26/06-09/01/06
12/16/06-12/22/06
week
An Overview of the Data
04/07/07-04/13/07
07/28/07-08/03/07
11/17/07-11/23/07
monica@mbarros.com
03/08/08-03/14/08
06/28/08-07/04/08
10/18/08-10/24/08
02/07/09-02/13/09
Natural Inflow Energy (Grande)
05/30/09-06/05/09
presents a distinct seasonal pattern.
09/19/09-09/25/09
Natural Inflow Energy - Paranaíba and Grande River Basins
12/09/10-01/15/10
05/01/10-05/07/10
08/21/10-08/27/10
12/11/10-12/17/10
04/02/11-04/08/11
14
Natural Inflow Energy in the Southeast Subsystem

15. Natural Inflow Energy (Average MW)
100.000
120.000
0
20.000
40.000
60.000
80.000
01/02/10-01/08/10
01/16/10-01/22/10
01/30/10-02/05/10
02/13/10-02/19/10
02/27/10-03/05/10
03/13/10-03/19/10
03/27/10-04/02/10
04/10/10-04/16/10
04/24/10-04/30/10
05/08/10-05/14/10
05/22/10-05/28/10
06/05/10-06/11/10
06/19/10-06/25/10
07/03/10-07/09/10
07/17/10-07/23/10
07/31/10-08/06/10
08/14/10-08/20/10
Natural Inflow Energy (SE)
08/28/10-09/03/10
week
09/11/10-09/17/10
09/25/10-10/01/10
10/09/10-10/15/10
10/23/10-10/29/10
11/06/10-11/12/10
An Overview of the Data
11/20/10-11/26/10
12/04/10-12/10/10
12/18/10-12/24/10
monica@mbarros.com
01/01/11-01/07/11
Spot Price (R$/MWh)
01/15/11-01/21/11
01/29/11-02/04/11
exhibit an inverse relationship
02/12/11-02/18/11
02/26/11-03/04/11
Spot Price and Natural Inflow Energy - 2010 and 2011
03/12/11-03/18/11
03/26/11-04/01/11
04/09/11-04/15/11
04/23/11-04/29/11
05/07/11-05/13/11
0
20
40
60
80
100
120
140
160
180
Spot price (R$/MWh)
Spot prices and Natural Inflow Energy (ENA)
15

16. The Model
Hybrid neural network/neuro-fuzzy
model for the spot price
Comments on the choice of the ANN Model
Comments on the NFS set-up
The structure of the hybrid model
monica@mbarros.com 16

17. The Model
The proposed model is a combination of a
backpropagation ANN and an ANFIS-type NFS.
In our model, the ANN forecasts are added to
the original inputs and fed to NFS to generate
the spot price forecasts.
Suppose the original ANN model contains n
inputs. The final “hybrid” model will contain
(n+1) inputs, the original ones plus an
additional input, obtained by “fitting” the ANN
to the dataset, generating one-step ahead
forecasts and adding the one-step ahead
forecasts as an additional input variable.
monica@mbarros.com 17

18. The Model
We created six different hybrid models.
Each model specializes in a single forecasting
horizon (one to 6 weeks ahead).
Comments on the Choice of the ANN Model:
The network structures for each model class
include an intermediate layer with a sigmoidal
activation function and an output layer with a
linear activation function.
ANNs with 6, 7, 8, 9, 10, 11 and 12 neurons in the
intermediate layer were tested.
monica@mbarros.com 18

19. The Model
Comments on the Choice of the ANN Model:
For each of these numbers of neurons, we tested
networks with 1000 to 3000 epochs.
The training period used for choosing the ANN
models was 90% of the data set.
One of the major issues regarding ANNs is the
dependence on the initial weights.
Due to this fact, the results produced by networks
with the same structure may vary considerably.
monica@mbarros.com 19

20. The Model
Comments on the Choice of the ANN Model:
In search of a more robust procedure, we replicate
the same network architecture several times and
chose the particular network that led to the smallest
one-step ahead MAPE in the training period.
We tested 21, 25, 31, 51, 75, 101, 121, 131, 151
replications of the same structure of several
different ANN models.
We chose to use 75 replications of each
architecture. In each, the ANN which produces the
best result (lowest MAPE).
monica@mbarros.com 20

21. The Model
Comments on the NFS Set-up:
As with the ANN model, several choices have to be
made regarding the specification of the NFS
implementation.
The neuro-fuzzy system with n inputs most often
outperformed the ANN with the same inputs.
The hybrid model consists of two steps:
1) Choose the “best” ANN with n inputs and record its one
step ahead forecasts;
2) Fit a NFS with the previous n inputs and an additional one,
the one step ahead forecasts obtain in the previous step.
The entire system requires a very modest amount
of information – just the past prices and past
natural inflow energy time series, which should be
updated weekly. monica@mbarros.com 21

22. The Model
The structure of the Hybrid Model:
the hybrid forecasting approach is a two step
procedure:
in the first step, 75 replications of a MLP neural
network with these inputs are adjusted and the best
network is selected, using as a criterion the
minimum MAPE during the training period
The second step employs the previously mentioned
inputs AND the forecasts generated by the best ANN
obtained in the first step as inputs in an ANFIS
neuro-fuzzy system
the objective of this model is to generate forecasts
up to six weeks in advance.
monica@mbarros.com 22

23. The Model
The structure of the Hybrid Model:
Let P(t) denote the price at week t, and suppose it
denotes the current week.
The forecast for P(t+1) uses as inputs the current
and lagged values of the spot prices and the natural
inflow energies at the subsystem and the basins,
namely:
P(t), P(t-2),
ENA_SE(t) (inflow energy of the subsystem at the current
week),
ENA_SE(t-1) (inflow energy of the subsystem one week ago),
ENA_GR(t-1) (inflow energy of the Grande river basin one
week ago),
ENA_PA(t) (inflow energy of the Paranaíba river basin at the
current week) .
monica@mbarros.com 23

24. The Model
The structure of the Hybrid Model:
It is necessary to forecast the input variables.
The forecasts of all natural inflow energy series
(ENA_SE, ENA_GR and ENA_PA) are obtained
exogenously through univariate time series models,
chosen to minimize the Bayesian Information
Criterion (BIC).
Natural inflow Durbin-
Model Structure R2 adjusted MAPE
energy series Watson
Southeast SARIMA(1,0,2)x(2,0,1) on ln of actual data 90.8% 12.1% 1.94
Paranaíba SARIMA(1,0,0)x(1,0,0) on ln of actual data 87.7% 17.4% 1.99
Grande SARIMA(1,0,2)x(2,0,1) on ln of actual data 85.6% 18.7% 1.92
monica@mbarros.com 24

25. Empirical Results
Southeast spot price six-step ahead forecasts at
the week Jan 1–7, 2011
Spot price six-steps ahead forecasting
at week Jan 1-7, 2011
FORECAST ACTUAL
60
52.2
50
Spot price (PLD) in R$/MWh
39.9
40
30.5 30.2
30 27.5
26.1
23.9
22.6
20 18.6
14.4
12.1 12.1
10
0
Jan 8-14, 2011 Jan 15-21, 2011 Jan 22-28, 2011 Jan 29-Feb 4, 2011 Feb 5-11, 2011 Feb 12-18, 2011
week
monica@mbarros.com 25

26. Empirical Results
Southeast spot price six-step ahead forecasts at
the week Feb 26–Mar 04, 2011
Six-steps ahead forecasting
at week Feb 26-Mar 04, 2011
40
36,61 35,92
35
Spot price (PLD) in R$/MWh
30
25
20
17,00
15 13,47
12,08 12,08 12,08 12,08 12,08 12,08 12,08 12,08
10
5
0
Mar 05-11, 2011 Mar 12-18, 2011 Mar 19-25, 2011 Mar 26-Apr 01, 2011 Apr 02-08, 2011 Apr, 09-15, 2011
FORECAST ACTUAL
monica@mbarros.com 26

27. Empirical Results
Error statistics of the six step ahead forecasts (%)
1 step 2 steps 3 steps 4 steps 5 steps 6 steps MAPE
Forecasts starting at week:
ahead ahead ahead ahead ahead ahead (week)
Jan 1-7, 2011 22.67 17.22 5.59 0.00 31.01 42.21 19.78
Jan 8-14, 2011 55.86 4.96 46.57 94.26 34.56 47.90 47.35
Jan 15-21, 2011 1.76 46.57 32.12 53.27 60.67 55.16 41.59
Jan 22-28, 2011 46.57 32.12 53.08 60.54 55.03 85.66 55.50
Jan 29-Feb 4, 2011 32.12 53.08 60.54 55.03 85.66 212.91 83.22
Feb 5-11, 2011 7.70 20.10 29.11 85.66 192.26 65.35 66.70
Feb 12-18, 2011 5.43 15.38 85.66 193.21 64.91 24.02 64.77
Feb 19-25, 2011 26.09 0.00 1.91 6.52 38.35 0.00 12.15
Feb 26-Mar 04, 2011 0.00 1.91 10.32 40.71 0.00 0.00 8.82
MAPE (forecast horizon) 22,02 21,26 36,10 65,47 62,49 59,25
monica@mbarros.com 27

28. Empirical Results
Forecasts produced in Actual values, Six-steps ahead forecasts
week April 9th-15th, at week Apr 9-15, 2011 and Inflow Energy time series
2011 and the following FORECAST ACTUAL Southeast Inflow Energy
50 120000
ones tend to be higher
than the actual values, 45 43,40
Southeast inflow Energy (MWmean)
39,79 100000
and sometimes the 40 38,69 38,16
Spot price (PLD) in R$/MWh
35,92
forecast errors are 35 34,00
80000
quite high, for no 30
26,16
apparent reason 25 60000
a possible explanation 20
15,62 15,64
17,09
40000
for this fact is that the 15 12,08
13,47
12,08 12,08 12,08 12,08 12,08
13,91
subsystem inflow 10
20000
energy has a 5
decreasing trend on 0 0
the weeks preceding March 5 - March 12 March 19 March 26 Apr 2-8, Apr 9-15, Apr 16-22, Apr 23-29, Apr 30- May 7-13, May 14- May 21-
11, 2011 -18, 2011 -25, 2011 -Apr 1, 2011 2011 2011 2011 May 6, 2011 20, 2011 27, 2011
April 9th-15th, 2011. 2011 2011
This behavior, in a hydro based system,
would lead to the dispatch of thermal plants
to save water and increase reservoir levels
resulting in an increase in the spot price.
monica@mbarros.com 28

29. Conclusions
The input variables considered are thought of
as important leading indicators of price
movements in a primarily hydroelectric system
such as Brazil’s
The forecasts produced were adequate most of
the time. However, in some instances, short-
term dispatch decisions affected prices in ways
that could not be anticipated by the model
monica@mbarros.com 29

30. Conclusions
The model can be improved further by
incorporating other variables, specifically those
related to thermal generation.
In fact, a trial neuro-fuzzy model has been
tested to forecast thermal generation, and the
forecast can be used as a “threshold” – if
above a certain amount, the forecast of the
original model need to be corrected upwards to
account for the dispatch of the thermal plants.
These results are, however, at a very
preliminary stage, so they were not reported
here.
monica@mbarros.com 30