Forecasting electricity demand distributions using a semiparametric additive model

1. Forecasting electricity demand distributions using a semiparametric additive model Rob J Hyndman Joint work with Shu Fan Forecasting electricity demand distributions 1

2. Outline 1 The problem 2 The model 3 Long-term forecasts 4 Short term forecasts 5 Forecast density evaluation 6 Forecast quantile evaluation 7 References and R implementation Forecasting electricity demand distributions The problem 2

3. The problem We want to forecast the peak electricity demand in a half-hour period in twenty years time. We have ﬁfteen years of half-hourly electricity data, temperature data and some economic and demographic data. The location is South Australia: home to the most volatile electricity demand in the world. Sounds impossible? Forecasting electricity demand distributions The problem 3

8. South Australian demand data Forecasting electricity demand distributions The problem 4

9. South Australian demand data Forecasting electricity demand distributions The problem 4

10. South Australian demand data Black Saturday → Forecasting electricity demand distributions The problem 4

11. The 2009 heatwave Forecasting electricity demand distributions The problem 5

14. The 2009 heatwave Average temperature (January−February 2009) 45 110 40 100 35 Degrees Fahrenheit Degrees Celsius 90 30 80 25 70 20 60 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Date in January/February 2009 Forecasting electricity demand distributions The problem 6

15. The 2009 heatwave Average temperature (January−February 2009) 45 110 40 100 35 Degrees Fahrenheit Degrees Celsius 90 30 80 25 70 20 60 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Date in January/February 2009 Forecasting electricity demand distributions The problem 6

16. The 2009 heatwave Average temperature (January−February 2009) 45 110 40 100 35 Degrees Fahrenheit Degrees Celsius 90 30 80 25 70 20 60 15 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Date in January/February 2009 Forecasting electricity demand distributions The problem 7

17. South Australian demand data Black Saturday → Forecasting electricity demand distributions The problem 8

18. South Australian demand data South Australia state wide demand (summer 10/11) 3.5 South Australia state wide demand (GW) 3.0 2.5 2.0 1.5 Oct 10 Nov 10 Dec 10 Jan 11 Feb 11 Mar 11 Forecasting electricity demand distributions The problem 8

19. South Australian demand data South Australia state wide demand (January 2011) 3.5 3.0 South Australian demand (GW) 2.5 2.0 1.5 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 Date in January Forecasting electricity demand distributions The problem 8

20. Demand boxplots (Sth Aust) Time: 12 midnight 3.5 3.0 2.5 Demand (GW) q q q q q q q q q q 2.0 q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 1.5 q q q q q q q q q q q q q q q q q q 1.0 q q Mon Tue Wed Thu Fri Sat Sun Day of week Forecasting electricity demand distributions The problem 9

21. Temperature data (Sth Aust) Time: 12 midnight 3.5 Workday Non−workday 3.0 2.5 Demand (GW) 2.0 1.5 1.0 10 20 30 40 Temperature (deg C) Forecasting electricity demand distributions The problem 10

22. Demand densities (Sth Aust) Density of demand: 12 midnight 4 3 Density 2 1 0 1.0 1.5 2.0 2.5 3.0 3.5 South Australian half−hourly demand (GW) Forecasting electricity demand distributions The problem 11

23. Industrial offset demand Winter Forecasting electricity demand distributions The problem 12

24. Industrial offset demand Summer Forecasting electricity demand distributions The problem 12

25. Outline 1 The problem 2 The model 3 Long-term forecasts 4 Short term forecasts 5 Forecast density evaluation 6 Forecast quantile evaluation 7 References and R implementation Forecasting electricity demand distributions The model 13

26. Predictors calendar effects prevailing and recent weather conditions climate changes economic and demographic changes changing technology Modelling framework Semi-parametric additive models with correlated errors. Each half-hour period modelled separately for each season. Variables selected to provide best out-of-sample predictions using cross-validation on each summer. Forecasting electricity demand distributions The model 14

35. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 yt denotes per capita demand (minus offset) at time t (measured in half-hourly intervals) and p denotes the time of day p = 1, . . . , 48; hp (t ) models all calendar effects; fp (w1,t , w2,t ) models all temperature effects where w1,t is a vector of recent temperatures at location 1 and w2,t is a vector of recent temperatures at location 2; zj,t is a demographic or economic variable at time t nt denotes the model error at time t. Forecasting electricity demand distributions The model 15

40. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 hp (t ) includes handle annual, weekly and daily seasonal patterns as well as public holidays: hp (t ) = p (t) + αt,p + βt,p + γt,p + δt,p p (t) is “time of summer” effect (a regression spline); αt,p is day of week effect; βt,p is “holiday” effect; γt,p New Year’s Eve effect; δt,p is millennium effect; Forecasting electricity demand distributions The model 16

46. Fitted results (Summer 3pm) Time: 3:00 pm 0.4 0.4 Effect on demand Effect on demand 0.0 0.0 −0.4 −0.4 0 50 100 150 Mon Tue Wed Thu Fri Sat Sun Day of summer Day of week 0.4 Effect on demand 0.0 −0.4 Normal Day before Holiday Day after Holiday Forecasting electricity demand distributions The model 17

47. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 6 + − fp (w1,t , w2,t ) = ¯ fk,p (xt−k ) + gk,p (dt−k ) + qp (xt ) + rp (xt ) + sp (xt ) k =0 6 + Fj,p (xt−48j ) + Gj,p (dt−48j ) j=1 xt is ave temp across two sites (Kent Town and Adelaide Airport) at time t; dt is the temp difference between two sites at time t; + xt is max of xt values in past 24 hours; − xt is min of xt values in past 24 hours; ¯ xt is ave temp in past seven days. Each function is smooth & estimated using regression splines. Forecasting electricity demand distributions The model 18

54. Fitted results (Summer 3pm) Time: 3:00 pm 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 Effect on demand Effect on demand Effect on demand Effect on demand 0.0 0.0 0.0 0.0 −0.2 −0.2 −0.2 −0.2 −0.4 −0.4 −0.4 −0.4 10 20 30 40 10 20 30 40 10 20 30 40 10 20 30 40 Temperature Lag 1 temperature Lag 2 temperature Lag 3 temperature 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 Effect on demand Effect on demand Effect on demand Effect on demand 0.0 0.0 0.0 0.0 −0.2 −0.2 −0.2 −0.2 −0.4 −0.4 −0.4 −0.4 10 20 30 40 10 15 20 25 30 15 25 35 10 15 20 25 Lag 1 day temperature Last week average temp Previous max temp Previous min temp Forecasting electricity demand distributions The model 19

55. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 Other variables described by linear relationships with coefﬁcients c1 , . . . , cJ . Estimation based on annual data. Forecasting electricity demand distributions The model 20

56. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 Other variables described by linear relationships with coefﬁcients c1 , . . . , cJ . Estimation based on annual data. Forecasting electricity demand distributions The model 20

57. Monash Electricity Forecasting Model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et J ¯ log(yi ) = cj zj,i + εi j =1 ¯ yi is the average demand for year i where t is in year i. ∗ yt is the standardized demand for time t. Forecasting electricity demand distributions The model 21

58. Monash Electricity Forecasting Model Forecasting electricity demand distributions The model 22

59. Monash Electricity Forecasting Model Forecasting electricity demand distributions The model 22

60. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD. Forecasting electricity demand distributions The model 23

61. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD. Forecasting electricity demand distributions The model 23

62. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD. zCDD = ¯ max(0, T − 18.5) summer ¯ T = daily mean Forecasting electricity demand distributions The model 23

63. Annual model ¯ log(yi ) = cj zj,i + εi j ¯ ¯ log(yi ) − log(yi−1 ) = cj (zj,i − zj,i−1 ) + ε∗ i j First differences modelled to avoid non-stationary variables. Predictors: Per-capita GSP, Price, Summer CDD, Winter HDD. zHDD = ¯ max(0, 18.5 − T ) winter ¯ T = daily mean Forecasting electricity demand distributions The model 23

64. Annual model and Heating degree days Cooling 600 Cooling and Heating Degree Days scdd 400 200 950 1050 whdd 850 1990 1995 2000 2005 2010 Forecasting electricity demand distributions The model 24

65. Annual model Variable Coefﬁcient Std. Error t value P value ∆gsp.pc 2.02×10−6 5.05×10−6 0.38 0.711 ∆price −1.67×10−8 6.76×10−9 −2.46 0.026 ∆scdd 1.11×10−10 2.48×10−11 4.49 0.000 ∆whdd 2.07×10−11 3.28×10−11 0.63 0.537 GSP needed to stay in the model to allow scenario forecasting. All other variables led to improved AICC . Forecasting electricity demand distributions The model 25

68. Annual model 1.7 Actual Fitted 1.6 1.5 Annual demand 1.4 1.3 1.2 1.1 1.0 89/90 91/92 93/94 95/96 97/98 99/00 01/02 03/04 05/06 07/08 09/10 Year Forecasting electricity demand distributions The model 26

69. Half-hourly models ∗ ¯ log(yt ) = log(yt ) + log(yi ) ∗ log(yt ) = hp (t ) + fp (w1,t , w2,t ) + et Separate model for each half-hour. Same predictors used for all models. Predictors chosen by cross-validation on summer of 2007/2008 and 2009/2010. Each model is ﬁtted to the data twice, ﬁrst excluding the summer of 2009/2010 and then excluding the summer of 2010/2011. The average out-of-sample MSE is calculated from the omitted data for the time periods 12noon–8.30pm. Forecasting electricity demand distributions The model 27

74. Half-hourly models x x1 x2 x3 x4 x5 x6 x48 x96 x144 x192 x240 x288 d d1 d2 d3 d4 d5 d6 d48 d96 d144 d192 d240 d288 x+ x− x dow hol dos MSE ¯ 1 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.037 2 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.034 3 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.031 4 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.027 5 • • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.025 6 • • • • • • • • • • • • • • • • • • • • • • • • • • • 1.020 7 • • • • • • • • • • • • • • • • • • • • • • • • • • 1.025 8 • • • • • • • • • • • • • • • • • • • • • • • • • • 1.026 9 • • • • • • • • • • • • • • • • • • • • • • • • • 1.035 10 • • • • • • • • • • • • • • • • • • • • • • • • 1.044 11 • • • • • • • • • • • • • • • • • • • • • • • 1.057 12 • • • • • • • • • • • • • • • • • • • • • • 1.076 13 • • • • • • • • • • • • • • • • • • • • • 1.102 14 • • • • • • • • • • • • • • • • • • • • • • • • • • 1.018 15 • • • • • • • • • • • • • • • • • • • • • • • • • 1.021 16 • • • • • • • • • • • • • • • • • • • • • • • • 1.037 17 • • • • • • • • • • • • • • • • • • • • • • • 1.074 18 • • • • • • • • • • • • • • • • • • • • • • 1.152 19 • • • • • • • • • • • • • • • • • • • • • 1.180 20 • • • • • • • • • • • • • • • • • • • • • • • • • 1.021 21 • • • • • • • • • • • • • • • • • • • • • • • • 1.027 22 • • • • • • • • • • • • • • • • • • • • • • • 1.038 23 • • • • • • • • • • • • • • • • • • • • • • 1.056 24 • • • • • • • • • • • • • • • • • • • • • 1.086 25 • • • • • • • • • • • • • • • • • • • • 1.135 26 • • • • • • • • • • • • • • • • • • • • • • • • • 1.009 27 • • • • • • • • • • • • • • • • • • • • • • • • • 1.063 28 • • • • • • • • • • • • • • • • • • • • • • • • • 1.028 29 • • • • • • • • • • • • • • • • • • • • • • • • • 3.523 30 • • • • • • • • • • • • • • • • • • • • • • • • • 2.143 31 • • • • • • • • • • • • • • • • • • • • • • • • • 1.523 Forecasting electricity demand distributions The model 28

75. Half-hourly models R−squared 90 R−squared (%) 80 70 60 12 midnight 3:00 am 6:00 am 9:00 am 12 noon 3:00 pm 6:00 pm 9:00 pm 12 midnight Time of day Forecasting electricity demand distributions The model 29

76. Half-hourly models South Australian demand (January 2011) 4.0 Actual Fitted 3.5 South Australian demand (GW) 3.0 2.5 2.0 1.5 1.0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 Forecasting electricity demand distributions Date in January The model 29

77. Half-hourly models Forecasting electricity demand distributions The model 29

78. Half-hourly models Forecasting electricity demand distributions The model 29

79. Adjusted model Original model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 Model allowing saturated usage J qt = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 qt if qt ≤ τ ; log(yt ) = τ + k(qt − τ ) if qt > τ . Forecasting electricity demand distributions The model 30

80. Adjusted model Original model J log(yt ) = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j =1 Model allowing saturated usage J qt = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 qt if qt ≤ τ ; log(yt ) = τ + k(qt − τ ) if qt > τ . Forecasting electricity demand distributions The model 30

81. Outline 1 The problem 2 The model 3 Long-term forecasts 4 Short term forecasts 5 Forecast density evaluation 6 Forecast quantile evaluation 7 References and R implementation Forecasting electricity demand distributions Long-term forecasts 31

82. Peak demand forecasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Multiple alternative futures created: hp (t ) known; simulate future temperatures using double seasonal block bootstrap with variable blocks (with adjustment for climate change); use assumed values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 32

86. Seasonal block bootstrapping Conventional seasonal block bootstrap Same as block bootstrap but with whole years as the blocks to preserve seasonality. But we only have about 10–15 years of data, so there is a limited number of possible bootstrap samples. Double seasonal block bootstrap Suitable when there are two seasonal periods (here we have years of 151 days and days of 48 half-hours). Divide each year into blocks of length 48m. Block 1 consists of the ﬁrst m days of the year, block 2 consists of the next m days, and so on. Bootstrap sample consists of a sample of blocks where each block may come from a different randomly selected year but must be at the correct time of year. Forecasting electricity demand distributions Long-term forecasts 33

93. Seasonal block bootstrapping Actual temperatures 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (fixed blocks) 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (variable blocks) Forecasting electricity demand distributions Long-term forecasts 34 40

94. Seasonal block bootstrapping Problems with the double seasonal bootstrap Boundaries between blocks can introduce large jumps. However, only at midnight. Number of values that any given time in year is still limited to the number of years in the data set. Forecasting electricity demand distributions Long-term forecasts 35

95. Seasonal block bootstrapping Problems with the double seasonal bootstrap Boundaries between blocks can introduce large jumps. However, only at midnight. Number of values that any given time in year is still limited to the number of years in the data set. Forecasting electricity demand distributions Long-term forecasts 35

96. Seasonal block bootstrapping Variable length double seasonal block bootstrap Blocks allowed to vary in length between m − ∆ and m + ∆ days where 0 ≤ ∆ < m. Blocks allowed to move up to ∆ days from their original position. Has little effect on the overall time series patterns provided ∆ is relatively small. Use uniform distribution on (m − ∆, m + ∆) to select block length, and independent uniform distribution on (−∆, ∆) to select variation on starting position for each block. Forecasting electricity demand distributions Long-term forecasts 36

100. Seasonal block bootstrapping Actual temperatures 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (fixed blocks) 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Bootstrap temperatures (variable blocks) 40 35 30 degrees C 25 20 15 10 0 10 20 30 40 50 60 Days Forecasting electricity demand distributions Long-term forecasts 37

101. Seasonal block bootstrapping Forecasting electricity demand distributions Long-term forecasts 37

102. Peak demand forecasting Climate change adjustments CSIRO estimates for 2030: 0.3◦ C for 10th percentile 0.9◦ C for 50th percentile 1.5◦ C for 90th percentile We implement these shifts linearly from 2010. No change in the variation in temperature. Thousands of “futures” generated using a seasonal bootstrap. Forecasting electricity demand distributions Long-term forecasts 38

110. Peak demand backcasting J qt,p = hp (t ) + fp (w1,t , w2,t ) + cj zj,t + nt j=1 Multiple alternative pasts created: hp (t ) known; simulate past temperatures using double seasonal block bootstrap with variable blocks; use actual values for GSP, population and price; resample residuals using double seasonal block bootstrap with variable blocks. Forecasting electricity demand distributions Long-term forecasts 39

111. Peak demand backcasting PoE (annual interpretation) 4.0 10 % 50 % 90 % 3.5 q q q PoE Demand q 3.0 q q q q q q q q 2.5 q q 2.0 98/99 00/01 02/03 04/05 06/07 08/09 10/11 Year Forecasting electricity demand distributions Long-term forecasts 40

113. Peak demand forecasting South Australia GSP 120 High billion dollars (08/09 dollars) Base 100 Low 80 60 40 1990 1995 2000 2005 2010 2015 2020 Year South Australia population 2.0 High Base Low 1.8 million 1.6 1.4 1990 1995 2000 2005 2010 2015 2020 Year Average electricity prices High 22 Base Low 20 c/kWh 18 16 14 12 1990 1995 2000 2005 2010 2015 2020 Year Forecasting electricity demand distributions industrial offset demand Long-term forecasts Major 42 0

114. Peak demand distribution Forecast density of annual maximum demand: 2009/2010 2.0 1.5 Density 1.0 0.5 0.0 2.5 3.0 3.5 4.0 4.5 5.0 Demand (GW) Forecasting electricity demand distributions Long-term forecasts 43

115. Peak demand distribution Annual POE levels 6 1 % POE 5 % POE 10 % POE 50 % POE 5 90 % POE q Actual annual maximum PoE Demand 4 q q q q 3 q q q q q q q q q 2 98/99 00/01 02/03 04/05 06/07 08/09 10/11 12/13 14/15 16/17 18/19 20/21 Year Forecasting electricity demand distributions Long-term forecasts 44

116. Peak demand forecasting Low Base 1.5 1.5 1.0 Density Density 1.0 0.5 0.5 0.0 0.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Demand (GW) Demand (GW) High 1.5 1.0 Density 2011/2012 2012/2013 2013/2014 2014/2015 0.5 2015/2016 2016/2017 2017/2018 2018/2019 2019/2020 0.0 2020/2021 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Demand (GW) Forecasting electricity demand distributions Long-term forecasts 45

117. Peak demand forecasting Low Base 100 100 80 80 60 60 Percentage Percentage 40 40 20 20 0 0 2.5 3.0 3.5 4.0 4.5 5.0 2.5 3.0 3.5 4.0 4.5 5.0 Quantile Quantile High 100 80 60 Percentage 2011/2012 40 2012/2013 2013/2014 2014/2015 2015/2016 2016/2017 20 2017/2018 2018/2019 2019/2020 2020/2021 0 2.5 3.0 3.5 4.0 4.5 5.0 Quantile Forecasting electricity demand distributions Long-term forecasts 45

118. Outline 1 The problem 2 The model 3 Long-term forecasts 4 Short term forecasts 5 Forecast density evaluation 6 Forecast quantile evaluation 7 References and R implementation Forecasting electricity demand distributions Short term forecasts 46

Forecasting electricity demand distributions using a semiparametric additive model

Recommended

Recommended

More Related Content

More from Rob Hyndman

More from Rob Hyndman (12)

Recently uploaded

Recently uploaded (20)

Forecasting electricity demand distributions using a semiparametric additive model