All of Virtual Bidding: A Data-Driven Approach

All of Virtual Bidding: A Data-Driven Approach
Wenyuan Tang1,2
joint work with
Ram Rajagopal2 Kameshwar Poolla1 Pravin Varaiya1
1University of California, Berkeley
2Stanford University
November 1, 2016
Wenyuan Tang Virtual Bidding 1 / 41

Two-Settlement Wholesale Electricity Market
Locational marginal prices (LMPs) reflect the value (price) of power
at different locations, and the LMP at a load-zone or a hub is the
weighted average of the nodal LMPs
The day-ahead (DA) market lets market participants commit to buy
or sell power one day before the operating day, and establishes 24
hourly DA LMPs
The real-time (RT) market balances the differences between DA
commitments and the actual demand and supply during the course of
the operating day, and establishes the 5-minute RT LMPs
Systematic nonzero spreads are routinely observed, which indicates
some market inefficiency
hourly spread = hourly DA LMP − hourly average RT LMP

Virtual Bidding
Virtual bids are included in DA dispatch, settled at DA LMPs,
liquidated at RT LMPs
Allows participants to take financial positions in DA without
delivering or consuming physical power in RT
Hedging tools for physical entities; arbitrage tools for financial
entities; adding liquidity; mitigating market power
Goals: enhancing market efficiency through DA/RT price convergence
(financial efficiency) and dispatch convergence (economic efficiency)
Virtual supply (INC): generation bid in DA to be closed in RT
Virtual demand (DEC): demand bid in DA to be closed in RT
DA RT
spread INC
DA RT
spread =
INC profit

Outline
Part I: Exploratory Data Analysis
Part II: Virtual Bidding and Financial Efficiency
Part III: Virtual Bidding and Economic Efficiency
Data
Analytics
Micro-
economics
Game
Theory
Empirical
Analysis
Virtual Bidding
Theory
Two-Settlement Market
Model
Financial
Efficiency
Economic
Efficiency

Part I: Exploratory Data Analysis
CAISO
Independent System Operator (ISO)
DA market: Apr 2009
Virtual bidding: Feb 2011
Data (NP15, 2010/2012): DA/RT LMP
Peak load ≈ 50 GW
PJM
Regional transmission organization (RTO)
DA market and virtual bidding: Jun 2000
Data (RTO, 2012–2015): DA/RT LMP,
DA/RT/forecast load, INC/DEC
Peak load ≈ 150 GW

DA and RT LMP
q
q q q q
q
q
q q q q q q q q
q
q
q
q
q q
q
q
q
q q q q q q
q q
q q q q
q
q
q
q
q
q
q
q
q
q q q
Mean
Standard Deviation
0
10
20
30
40
50
0
10
20
30
40
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour
LMP($/MWh)
Market
q DA
RT
Statistics of DA and RT LMP, PJM, 2012
spread = DA LMP − RT LMP ≈ 0
RT LMP is more volatile than DA LMP

Spread
−200
−100
0
100
200
Mar Jun Sep Dec
Spread($/MWh)
Hour
4
18
Hourly Spread Time Series, PJM, 2012
0.00
0.05
0.10
−200 −100 0 100 200
Spread ($/MWh)
Density
Hour
4
18
Hourly Spread Histogram, PJM, 2012
The distribution of the spread is heavy-tailed and left-skewed

Spread
−500
−250
0
250
Mar Jun Sep Dec
Spread($/MWh)
Hour
4
18
Hourly Spread Time Series, PJM, 2014
The polar vortex triggered two extreme weather events in Jan 2014
Recent data do not support classical models, e.g., [Bessembinder &
Lemmon 2002], which states that spread is negatively related to
Var(RT LMP), and positively related to Skew(RT LMP)

Virtual Bids
q
q
qqq
q
qq
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
qq
q
q
q
q
qq
q
qqq
q
qqqqq
q
q
q
q
q
qqq
q
qqqq
qq
qqq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
qqqq
q
qq
q
q
qq
q
qq
q
qq
q
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qqq
q
qq
q
qqqq
q
q
qq
qqq
q
q
q
q
q
qq
qq
q
q
q
q
q
q
qq q
qq
q
q
qqq
qq
qq
q
q
qq
q
q
q
qq
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
qq
q
q
qqq
qq
qq
q
q
qq
q
qq
q
q
qq
qqq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
qqq
q
q
qq
q
qqq
q
qq
q
q
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
qqq
qq
q
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
qqq
q
q
q
qq
q
q
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qqqq
q
q
q
q
q
q
q
q qqqq
q
q q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
qqq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
qq
qq
qq
q
q
qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qqqqq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
qqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
q
q
qqqqq
qq
q
q
q
q
qq
qq
qqq
qq
q
qq
q
qq
qq
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
qq
qqq
q
q
qqq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
qqqqqq
q
q
qq
qqq
q
qq
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qqq
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q q
q
q
q
qq
qq
q
q
qq
q
qq q
qq
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
q
q qq
q
q
q
q
qqq qq
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
qqqq
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
qq
q
q
qq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q q
q
qq
q
qq
q
q
q
q
q
q
q
qqq
q
q
q
q
qqq
qqq
q
q
q
qq
qq
qqq
q
qq
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
qq
qqq
qq
q
q
q
q
q
q
qq
qqq
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
qq
qq
q
q
q
qq
q
q
qq
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
q
qq
qq
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qqq
q
qq
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
qqq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
qqqq
q
q
q
qq
q
q
q
q q
q
qq
q q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
qqq
q
q
q
q
qq
q
qq
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
qq
q
q
q
q
qq q
q
q q
q
q
q
q
q q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
qqq
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
qq
q
q
q
q
q
q
q qqqq
q
q
q
q
qq
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
qq
q
q
q
qqqq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
qqqq
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qqq
q
qqq
q
qq
qq
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
qq
q
q
q
qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
q
q
qqq
q
qq
q
q
qqqqq
q
q
qq
qq
q
q
qq q
q
qq
q
q
q
qq
q
q
qqq
q
qq
q
qqq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
q
qqq
qq
−15000
−10000
−5000
0
5000
−400 −200 0 200
Spread ($/MWh)
NetINCVolume(MW)
q Profit
Loss
Hourly Net INC Volume versus Spread, PJM, 2012
proﬁt = spread × (INC − DEC) = spread × net INC
Virtual bidders expect negative spreads, or RT LMP spikes

Virtual Bids
2012 2013
2014 2015
($20)
$0
$20
$40
($20)
$0
$20
$40
Mar Jun Sep Dec Mar Jun Sep Dec
Mar Jun Sep Dec Mar Jun Sep Dec
CumulativeProfit(Millions)
Hours
All
Normal
Abnormal
Profit of Virtual Bids, PJM
Normal hours (98% of all): spread between 1st and 99th percentile
Abnormal hours (2% of all): otherwise

Performance Metric: Sharpe Ratio
N: number of days, R: daily proﬁt
Sharpe ratio =
√
N
E[R]
Var(R)
Four-year (2012–2015) Sharpe ratio of S&P 500 is 1.68
Sharpe ratios of the PJM virtual bids
Year All Hours Normal Hours Abnormal Hours
2012 1.80 0.40 1.99
2013 0.96 −2.09 2.77
2014 0.39 −1.42 0.93
2015 4.31 1.60 4.82
Total 1.79 −1.29 2.56
Virtual bidders speculate on extreme events

We define financial efficiency as DA/RT price convergence
DA LMP = E[RT LMP|DA LMP]
or
E[spread|DA LMP] = 0
which implies
ρ(spread, DA LMP) = 0 and E[spread] = 0
It is not clear that virtual bidding improves the financial efficiency
q
q q
q
q
q
q
q q
q
q q
q
q
q
q q q
q
q q q
q
q
−0.2
−0.1
0.0
0.1
0.2
0.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour
Correlation
Year
q 2010
2012
Correlation between Spread and DA LMP, CAISO

q
q
q
q
q q
q
q
q q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q q
q q q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q q
q
q q q
q q
q q
q
q
q q
q
q q
q q q
q
q
q
q
Mean
Standard Deviation
Mean Absolute Value
−10
−5
0
5
10
0
20
40
60
80
0
10
20
30
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour
Spread($/MWh)
Year
q 2010
2012
Statistics of Spread, CAISO
Mean is closer to zero after the implementation of virtual bidding
But standard deviation and mean absolute value remain high

Alternative approach: testing whether profitable bidding strategies
exist before and after the implementation of virtual bidding [Li,
Svoboda & Oren 2015], [Jha & Wolak 2015]
We propose a measure that tests the randomness of the sequence of
the spread: more random spread leaves less room for arbitrage
opportunities
We examine the autocorrelation of the sequence of the spread and
propose a benchmark bidding strategy that is only based on the
up-to-date price information
We employ machine learning methods to design more sophisticated
bidding strategies that utilize other data such as load
Note that our definition of financial efficiency does not depend on
transaction costs, which are therefore not considered

Wald-Wolfowitz Runs Test on sgn(spread)
qqqqqqqqqqq
q
qqqqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
q
qqqqq
qqqqqqqqqqqqqqqqq
qqqqqqqqqq
qqq
qqqqqqqq
qqqqqqqqqqqqqqq
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
qqqq
qqqqqqqq
qq
−400
−300
−200
−100
0
100
Mar Jun Sep Dec
Spread($/MWh)
Spread
q > 0
< 0
Runs Test on Sign of Spread, CAISO, 2010, Hour 18
A run is a segment of consecutive “+”s or “−”s
N = 365 = N+ + N− = 268 + 97. Under the null hypothesis that the
sequence is i.i.d.
µ = (2N+N−/N) + 1 = 143, σ = (µ − 1)(µ − 2)/(N − 1) = 7.4
110 runs, few enough to reject the null hypothesis (at α = 0.05)

Runs Test Results (p-values ≤ 0.05 in bold)
Hour 2010 2012 Hour 2010 2012
1 0.016 0.331 13 0.000 0.074
2 0.026 0.028 14 0.000 0.123
3 0.161 0.020 15 0.002 0.002
4 0.002 0.625 16 0.000 0.002
5 0.063 0.352 17 0.002 0.002
6 0.050 0.998 18 0.000 0.001
7 0.000 0.561 19 0.000 0.033
8 0.021 0.730 20 0.000 0.000
9 0.016 0.501 21 0.006 0.323
10 0.524 0.066 22 0.000 0.000
11 0.000 0.001 23 0.000 0.005
12 0.000 0.001 24 0.002 0.703
The spread is more random after virtual bidding

Exploring Intertemporal Correlation
95% CI
95% CI
2010
2012
−0.1
0.0
0.1
0.2
0.3
−0.1
0.0
0.1
0.2
0.3
0 5 10 15 20 25
Lag (Day)
PACF
Partial Autocorrelation of Daily Average Spread, CAISO
The spread is more random after virtual bidding
The strong lag-1 autocorrelation motivates the benchmark strategy

Lag-1.5 Algorithm
sgn( spread) sgn( spread)
forecast
deadline
a.m. p.m. a.m. p.m. a.m. p.m.
day t − 2 day t − 1 day t
Bids for day t should be submitted by noon on day t − 1
Lag-1.5 forecast (sh
t : spread at hour h on day t)
sgn
24
h=13
sh
t−2 +
12
h=1
sh
t−1 → sgn
24
h=1
sh
t
If “+”, trade 1 MW INC for each hour
If “−”, trade 1 MW DEC for each hour

Lag-1.5 Algorithm
2010
2012
($5,000)
$0
$5,000
$10,000
($5,000)
$0
$5,000
$10,000
Mar Jun Sep Dec
Mar Jun Sep Dec
CumulativeProfit
Hours
All
Normal
Abnormal
Profit of the Lag−1.5 Algorithm, CAISO
Pre-VB: average proﬁt $1.35/MWh, Sharpe ratio 1.93
Post-VB: average proﬁt $1.08/MWh, Sharpe ratio 1.35

Support Vector Machine
Input variables: {spread, DA load, RT load, forecast load} of day
t − 7, t − 6, . . . , t − 2
Output variable: sign of the daily spread of day t
Training data: 365 samples in year y
Test data: 365 samples in year y + 1
Sharpe ratios of the PJM virtual bids and SVM
Year
PJM SVM
All Normal Abnormal All Normal Abnormal
2013 0.96 −2.09 2.77 1.66 4.72 −1.26
2014 0.39 −1.42 0.93 1.62 3.19 0.98
2015 4.31 1.60 4.82 1.29 3.75 −0.81
Total 1.19 −1.60 1.97 2.17 6.05 0.63
There is still room for arbitrage opportunities: even a simple machine
learning algorithm works well

Support Vector Machine
2013 PJM
2014 PJM
2015 PJM
2013 SVM
2014 SVM
2015 SVM
($10,000,000)
$0
$10,000,000
$20,000,000
($20,000,000)
$0
$20,000,000
$40,000,000
$0
$10,000,000
$20,000,000
($2,500)
$0
$2,500
$5,000
$7,500
$0
$10,000
$20,000
$30,000
($3,000)
$0
$3,000
$6,000
$9,000
Mar Jun Sep Dec
Mar Jun Sep Dec
Mar Jun Sep Dec
Mar Jun Sep Dec
Mar Jun Sep Dec
Mar Jun Sep Dec
CumulativeProfit
Hours
All
Normal
Abnormal
Profit Comparison between PJM Virtual Bids and SVM
SVM capture the patterns: proﬁtable in normal hours
Actual virtual bidders speculate on the extreme events

Part III: Virtual Bidding on Economic Efficiency
We define economic efficiency as generation cost minimization
Why is load forecast important?
Why is DA load close to RT load?
Why is RT supply curve steeper than DA?
How is economic efficiency related to price convergence?
How is economic efficiency related to dispatch convergence?
How to explain the phenomena: negative RT LMP; DA load close to
RT load, but DA LMP far apart from RT LMP; etc.
How does virtual bidding affect DA/RT dispatch?
How to estimate the generation cost with and without virtual bids?

DA/RT Load
q
q
q q q
q
q
q
q
q
q
q q q q q
q
q q q q
q
q
q
q
q
q q
q
q
q
q
q
q
q q q q q q
q
q q q q
q
q
q
q
q
q q
q
q
q
q
q
q
q q q q q q
q
q q q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q q q q
q
q q q q
q
q
q
2012 2013
2014 2015
70
80
90
100
70
80
90
100
0 6 12 18 24 0 6 12 18 24
Hour
Load(GW)
Market
q DA
RT
Mean of DA and RT Load, PJM
DA load is close to RT load

DA/RT Generation
q
q
q q q
q
q
q
q
q
q
q q q q q
q
q q q q
q
q
q
q
q
q q
q
q
q
q
q
q
q q q q q q
q
q q q q
q
q
q
q
q
q q
q
q
q
q
q
q
q q q q q q
q
q q q q
q
q
q
q
q
q q
q
q
q
q
q
q
q q q q q q
q
q q q q
q
q
q
2012 2013
2014 2015
70
80
90
100
70
80
90
100
0 6 12 18 24 0 6 12 18 24
Hour
Generation(GW)
Market
q DA
RT
Mean of DA and RT Generation, PJM
DA generation is close to RT generation
DA generation may not equal DA load due to virtual bids

DA/RT LMP and Generation
q q q qqqqq q qqq q qqqqq qqqqq q qqq q q qqqq qqqq q q qqq qqqqqqqq q qqqq q qqqqq qq q q qqq q qqqqqqqqqq qqqqqqq qqq qqqq qq qqqqq q qqqqq q qqqqqq q qqqqqqqq q qqqqq qqqqqq q q qqqqq q qq q q qq q qqqqqq qqqq qqq qqqq q qqq q q qqqq qqq q qqq qqq q qqqqqqq qqq q q qqqqq q qq q qqq q q qq qqqqq qqqqqq qqqqqq q qqq qqqq qqq q qqq q qqqqqqqq qqqq q qqq qqq qq q qqqqq q qqqqqq qq qqqqqq qq qqqq qqq qqq qq qqqqq q qq qqqqq q qqqqqq qqqq q qqq qqqqqqqq qqqqq q q qqqqq q q qq qqqq q q qqq qq q q qqqqq q qqq q qqqqq qqqqq q qqq q q qqqq qqq qq q qqq qqqqqqqq q qqqq q qqqqq qq q q qqq q qqqqqqqqq q qqqqqq q qqq qqqq qq qqqqq q q qqqq q qqqqqq q qqqq qqqq q qqqqq qqqqqq q q qq qqq q qq q q qqq qqq qqq qqqq qqq qqqq q qqq q q qqqq qqq q qqqqqq q qqqqqqq qq q q q qqqqq q qq q qqqq qqq qqqqq qqqqq q qqqqqq q qq q qqq q qqq q qqq q qqqqqq qq qqqq q qqqqqq q q q qqqqq q qqqqq q qqq qqqq q qq qqq q qqqqqq qq qqqqq q qq qqqqq q qqqqqq qqqq q qqq qqqqqqq q qqqqq q q qqqqq q q qq qqqq qq qqqqq q q qqqqq q qqq q qqqqq qqqqq q qqq q q qqqq qqq q q q qqq qqqqqqqq q qqqq q qqqqq qq q q qqq q qqqqqqqqq q qqqqqq q qqq qqqq qq qqqqq qq qqqq q qqqqqq q qqqq qqqq q qqqqq qqqqqq q q qqqqq qqqq qqq q qqq qqq qqqq qqq qqqqqq q q q q qqqq qqq q qqq qqq q qqqqqq q qq q q q qqqqq q qq q qqq q qqq qqqqq q qqqq q qqqqqq q qq q qqq q qqq q qq q q qqqqqq qq qqqq q qqqqqq qq qqqqqq q qqqqq q qq q qqqq q qq qqq qqqqqqq qq qqqqq q qq qqqq q q qqqqq q qqqq q qqq qqqqqqq q qqqqq q q qqqqqq q qq qqqq qq qqq qq q q qqqqq q qqq q qq qqq qqqqq q qqq q q qqqq qqq qq q qqq q qqqqqqq q qqqq q qqqqq qq q q qqq q qqqqqqqqq q qqqqqq q qqq qqqq qq qqqqq q q qqqq q qqqqqq q qqqq qqqq q qqqqq qqqqqq q q qq qqq qqq q qqqq qqq qqq qqqq qqq qqqqqq q q q q qqqq qqq q qqq qqq q qqqqqq q qq q q q qqqqq q qq q qqq q qqq qqqqq q qqqq q qqqqqq q qq q qqq q qqq q qq q q qqqqqq qq qqqq q qqqqqq qqqqqqqq q qqqqq q qq q qqq q q qq qqq qqqqqqq qq qqqqq q qq qqqq q q qqqqq q qqqq q q qq qqqqqqq q qqqqq q q qqqqq q q qq qqqq qq qqq qq q q qqqqq q qqq q qq qqq qqqqq q qqq qq qqqq qqq qq q qqq q qqqqqqq q qqqq q qqqqq qq q q qqq q qqqqqqqqq q qqqqqq q qqq qqq q qq qqqqq q q qqqq q qqqqqq q qqqqqq qq q qqqq q qqqqqq qqqqqqq qqqq qqq q qqq qqq qqqq qqq qqqqqq q q q q qqqq qqq q qqq qqq q qqqqqq q qqq q q qqqqq q qq qqqq q qqq qqqqq q qqqq q qqqqqq q qq q qqq q qqq q qq q q qqqqqq qq qqqq q qqqqqq qqqqqqq q q qqqqq q qq q qqq q q qq qqq qqqqqqq qq qqqqq q qq qqqq q q qqqqq qqqqqq q qq qqqqqqq q qqqq q q q qqqq q q qqq qqqq qq qqq qq q q qqqqq q qqq q qq qqq qqqqq q qqq q q qqq q qqq qq q qqq q qqqqqqq q qqqq q q qqqq qq q q qqq q qqqqqqqqq q qq qqqq q qqq qqq q qq qqqq q qq qqqq q qqqqqq q qqqqqq qqq qqqq q qqqqqq qq qqqqq qqqq qqq q qqq qqq qqqq qq q
qqqqqqq q q q qqqq qqq q qqq qqq q qqqqqq qqqq q q qqqqq
q qq q qqq q qqq qqq qq qqqqq q qqqqqq q qq q qqq q qqq qqqq q qqqqq q qq qqqq q qqqqqq qq q qqqq q q qqqqq q qq qqqq q q qq qqq qqqqqqq qq q qqqq q qq qqqq q q qqqqq qqqqqq q qq q qqqq qq q qqqq q q q qqqq qq qqqqqqq qq qqq qq q
q q
qqqq q q qq qqq qq q qq
qq q q qqqqq
qqq q qqq
qq q qqq q
q
qqqqqq q qqqqq q qqqq qq q
q qq
q qqq qqqqqqq q qqqqqq q
q
qq qqq q qq qqqq q q q qqqq q qq qqqq q qqqqqq qqq qqqq q qqqqqq qq qqqqq qq qqqqq q qqq qqq qqqq qq q
qqqqqq q
q q q qqqq
qqq q qqq qqq q qqq qqq qqqq q q qqqqq
q qq q qqq
q qqq qqq qq qqqqq q qqqqqq q qq q qqq q qqq qqq q q q qqqq qqq qqqq q qqqqqq qq qqqqq q q qqqqq
q qq q qqq q q q qqqq
qqqq qqq
qq
qq q
qq
q qq qq
qq q q qqq
qq qq qq qq q
qq qqqqq qq q qqqq q q qq qqq qq qqqqqqq q q qqq qq q
q
q
qqqq qq qq qqq qq q
q q
qq q q
q
qqqq
qqqq qqq
qqq qqq q
q
qqqqqq q qqqqq q
qqqq qq q
q q
q
q qqq qqqqqqq q qqqqqq q
q
qq qqq q qqqqqq q q q q q
qq qqqqqqq q qqqqqq qq q qqqq q qqqqqq qq qqqqq qq qq qqqq qqq qqq qqqq qqq qqqqqqq q q q qqqq qqq q qqq qqq qqqq qqq qqqq q q qqqqq q qq qqqq q qqq qqq qqq qqqq qq qqqqq qqq qqqq q qqqqqqq q q qqqq qqqqqqq q qqqqqq qq qqqqq q q qqqqq
q qq q qqq qq q qqqq
qqqqqqq
qq
qqq
qq
q q qqq
qq
q q
qqq
qq qqq
q qq q
qq q qqqq qqq
qqqq q q q qqqq qq qq qqqqq q q qqq qq q
q qqqqq qq qq qqq qq q qqqq q q qqqqq qqqq qqq qq q qqq q qqqqqqq q qqqqqq qqqq qq q q qqq qqq qqqqqqq q qqqqqq q qqq qqq q qqqqqq qq qq qqq qqqqqqq qqqqqqq q q q qqqq q qqqqqq qq qqqqq qq qqqqqq qqq qqq qqqq qqq qqqq qqq q q q qqqq qqq q qqq qqq qqqq qq q qqqq q q qqqqq q qq qqqq q qqq qqq qqq qqqq qq qqqqq qqq q qqq qqqq qqq q q qqqqq qqqqqqq q qqqqqq qq qqqqq q q qqqqq
q qq qqqq qqqqqqq qqqqqqq qq
qqqqq q qqqqqq q q qqqqq qqqq q q q
qq qqqqq qq q qqqq q q qq qqq qqq q qqqqq q qqqq qq q
qqqqqq qqqq qqq qqq qqqqq q qqqqq qqqq qqq qq qqqq q qqqqqqq q qqq qqq qqqqqq q q qqq qqq qqqqqqq q qqq qqq q qqqqqq q qqqqqq qq qqqqq qqqqqqq qqqqqqq q q qqqqq q qqqqqq qq qqqqq qq qq qqqq qqq qqq qqqq qqq qqqq qqq q q q
q
qqq qqq q qqq qqq qqqqqqq qqqq q q qqqqq q qq qqqq qqq q qqq q qq qqqq qqqqqqq qq q q qqq qqq q qqq q q qqqqq qqqqqqq q qqqqqq qq qqqqq q q q qqqq q qq qqqq qqqq qqq qqqqqqq qq
qqqqq q q qqqqq q qqqqqq qqqq q q q qqqqqqq qq q qqqq q q qqqqq qqq q qqqqq qqqqq qq q qqqqqq qq qq qqq qqq qqqq q q qqqqq qqqq qqq qq qqq qq qqqqqqq q qqq qqq qq qqqqq q qqq qqq qqq qqqq q qqq qqq q qqqqqq qqqqqqq qq qqqqq qqqqqqq qqq qqqq q q q qqqq q qqqqqq qq qqqqq qq q q qqqq qqq qqq qqq q qqq qqqq qqq q q q
q
qqq qq q q qqq qqq qqqqqq q qqqq q q q
qqqq q qq qqqq qqq q qqq q qq qqqq qqq qqqq qq q q qqq q qq q qqq q q qqqqq qq qqqqq q qqq qqq qq qqqqq q q qqqqq q qq qqqq qqqqqqq qqqqqqq qq
qqqqq q q qqqqq q qqqqqq qqqq q qq qqqqqqq q qq qqqq q q qqqqq qqq q qqq qq q q qqq qq q qqqqqq qq qq qqq qqq qqqq q q qqqqq qqqq qqq qq qqq qq qqqqqq q q qqq qqq qq qqqqq q qqq qqq qqq qqqq q qqq qqq qqqqqqq qqqqqqq qq qqqqq
qqq qqqq qqq qqqq q q q qqqq q qqqqqq q q qqqqq qq q q qqqq qqq qqq qqq q qqq qqqq qqq q q
q
q
qqq qq q q qqq qqq qqqqqq q qqqq q q
q
qqq q
q qq
qqqq qqq q qqq q qq qqqq qqq qqqq qq q q qqq q qq q qqq q q q qqqq qq q qqqq q qqq qqq q q qqqqq q q qqqqq q qq qqqq qqqq qqq q qqqqqq qq qqqqq q q qqqqq q qqqqqq qqqq q qq q qqqqqq q qq qqqq q q qqqqq qqq q qqq qq qqqqq qq q qqqqqq qq qq qqq qqq qqqq q qqqqqq qqqq qqq qq qqq qq qqq qqq q q qqq qqq qq qqqqq q qqq qqq qqq qqqq q qqq qqq qqqqqqq qqqqqqq qq qqqq q
qqq qqqq qqq qqqq q q q qqqq q qqqqqq q q qq qqq qq q q qqqq qqq qqq qqq q qqq qqqq qqq q q
q
q
qqq qq q
q qqq qqq
qqqqqq q qqqq q
q
q
qqq q
qqq
q
q
qq qqq q q
qq q q q qqqq qqq qqqq q q q q qqq qqq q qqq q q q qqqq qq qqqqq q qqq qqq qq qqqqq q q qqqqq q qq qqqq qqqqqqq q q qqqqq qq qqqqq q q qqqqq q qqqqqq qqqq q qq q qq qqqq q qq qqqq q qqqqqq qqq q qqq qq qqqqq qq q qqqqqq qq qq qqq qqq qqqq q qqqqqq qqqq qqq qq qqq qq qqq qqq q q qqq qqq qq qqqqq qqqq qqq qqq qqqq q q qq qqq qqqqqqq qqqqqqq qq qqqq q
q
qq qqqq qqq qqqq q q q qqqq q qqqqqq qq qq qqq qq q q qq qq qqq qqq qqq q q qq qqqq qqq q q
q
q
q
qq qq q
q
q
qq qqq
q
q
q
qqq q qqqq
q
q
q
qqq q
qq
q
q
q
qq qqq q
q
qq q q q qqq q qq q qqqq q q q q qqq qqq q qqq q qq
qq
qq qq qqqqq q qqq qqq q q qqqqq q q qqqqq q qq qqqq qqqqqqq q qqqqqq qq qqqqq q q qqqqq q qqqqqq qqqq q qq q qqqqqq q qq qqqq q qqqqqq qqq q qqq qq qqqqq qq q qqqqqq qq qq qqq qq q qqqq q qqqqqq qqqq qqq qq qqq qq qqq qqq qq qqq qqq qq qqqq q qqqq qqq qq q qqqq q q qq qqq qqqqqqq qqqqqqq qq qqqq q
q
qq qqqq qqq qqqq q q q qqqq q qqqqqq q q qq qq q qq q q q q q q q
qq qqq qqq q qq q qqqq qqq q q
q
q
q
qq qq q
q
q
qq qqq
qq
q
qqq q qqq q
q
qq
q
qq q
qq
q
q
q
qq qqq q
q
qq q qq q
qq q qq q qq
qq q q q q qqq q qq q
q
qq q
qq qq
qq q q qqqqq q qqq qqq q q qqqqq q q qqqqq q qq qqqq qqqqqqq q q qqqqq qq qqqqq q q qqqqq q qqqqqq qqqq q qq q qqqqqq q q q qqqq q qqqqq q qqq q qqq qq qqqqq qq q qqqqqq qq qq qqq qq q qqqq q qqqqq q qqqqqqq q q qqq qq qqq qqq qq qqq qqq qq qqqq q qqqq qqq qq q qqqq q q qq qqq qqqqqqq qqqqqqq qqqqqq q
q
qq qqqq qqq qqqq q q q qqqq q qqqqqq q q qq qq q qq q q q qqq q
qq qqq qqq q qq q qqqq qqq q
q
q
q
q
qq qq q
q
q
qq qqq
qq
q
qqq q qq
q q
q
qq
q
qq q
qqq
q
q
qq
qqq q
q
qq q qq q
qq q
qq q qq
qq q q q q qqq
q qq q
q
q
q q
qq
qq
qq q q qqq
qq q qqq qq
q q q qqqqq q q qqqqq q qq qqqq qqqqqqq q q qqqqq qq qqqqq q q qqqqq q qqqqqq qqqq qqq q qqqqqq q q q qqq q q qqqqq q qqq q qqq qq qqqqq qq q qqqqqq qq qq qqq qq q qqqqq qqqqq q qqqqqqq q q qqq qq qqq qqq qq qqq qqq qq qqqq q qqqq qqq qq q qqqq q q qq qqq qq qqqqq qqqqqqq qq qqqq q
q
qq qqqq qqq qqqq q q q qqqq q qqqqqq q q qq qq q qq q q q qq q q
qq qqq qqq q q q q qqqq qqq
q
q
q
q
qqq qq
q
q
q
qq qqq
qq
q
q
qq q qq
q q
q
q
q
q
qq q
qqq
q
q
qq qqq
q
q
q
q q
qq q
qq q qq q qq
qq q q q q q
qq
qqq
q
q
q
q q
q
q q
q
qq q q qqq
qq qqqq qqq q q qqqqq q q qqqqq q qqqqqq qqqqqqq q q qqqqq qq qqqq q q q qqqq q q qqqqqq qqqq q qq q qqqqqq q q q qqq q q qqqqq q qqq q qqq qq qqqqq qq
q
q
qqqq q qq qq
qqqqq qqqq qq qqqqqq qqqqqq q q q qqq q qqqq qqq qqqqq qqq qq qqqq q qqqq qqq qqq qqqq q q qqqqq qqqqqqq qqqqqqq qq qqqq q
q
qq qqqq qqq qqqq q q q qqqq qqqqqq q q q qq qq q qq q q q qq q qqq qqq qqq q q q q qqq q qqq
q q
q
q
qq q qq
q
q
q
qq
qqq
qq
q
qqq q qqq q
q
q
q
qqq q
qqq
q
qqq qqq
qq
qq q qq qqq q qq q qqqq q q q q qqq qqq
q
q
qq q qq q
q
qq q q qqqqq qqqq qqq q q qqqqq q q qqqqq q qqqqqq qqqqqqq q q qqqq q qq qqqq
q
q q qqqq q q qqqqqq qqq
q q q q
qqqqqqq qq q q
qq q q q qqqqq qqq q qqqqq qqqqq qq
q
q
qqqq q qq qq
qq qqq qqqq qq qqqqq q qqqqqqq q q qqq
qq qqqqqq q qqqq qqq qq qqqq q qqqq qq
q qqq qqqq q q qqqqq qqqqqqq qqqqqqq qq qqqq q qqq qqqq qqq qqqq q q q qqqq qqqqqq q q q qq qq q qq q q q q q q qqq qq q qqq q q q q qqq q qqq q q
q
q
qq qqq q q q
qq qqq
qqq
qqq q qqq q
q
q
q
qqq q
qqq
q
qqq qqq qq
qq q qqqqq q qq q qqqq q q q q qqq qqq
q q
qq q q q qqqq q q qqqqq qqqqqqq q q qqqqq q q qqqq q q qqqqqq qqq qqq q qq qqqq q qqq
qqq q q q qqqqq q qqqqqq qqqq q qq qqqqqqq qq q qqq q q q qqqqq q qq q qqqqq q qqqqqq
q
q
qqqq q qq qq qq qqq qqqqqq qqqqq q qqq qqqq qq qqq q qqqqqqq q qqqq qqq qqqqqq q qqqq qqq qqq qqqq q q qqqqq qqqqqqq qqqqqqq qq qqqq q q
qq qqqq qqqqqqq q q q qqqq qqqqqq q q q qq qq q qq q qq q q q qqq qq q qqq q q q q qqq q qqq q q
q
q
qqqqq q qqqq qqq
qqq
qqq q qqq q q
qq
qqq q qqq q
qqq q qq q qqq q qqqqq q qq q qqqq q q q q qqq qqq q qqq q q q qqqq q q qqqq q qqqq qqq qq qqqqq q q qqqqq
q
qqqqqq qqq qqqq qq qqqq q qq qqqq q q q qqqqq q qqqqqq qqqq q qq qqqqqqq qq q qqq q q q qqqqq q qq q qqqqq q qqq qqq q q
qqqq q qq q q qq qqq qqqqqq qqqqq q qqq qqqq q q qqq q qqqqqqq q qqqq qqq qqqqqq q qqqq qqq qq q qqqq q q qqqqq q qqqqqq qqqqqqq qq qqqq q q
qq qqqq qqq qqqq q q q qqqq qqqqqq q qq qq qq q qq q qq qq qqqq qq q qqq q q q q qqqq qqq q q
q q
qqq qq q qqqq q qq qq qqqq q qqq q q
qq
qqq q qq q q
qqq qqq qqqq q qqqqq q qq q qqqq q q q q qqq qqq q qqq q q q qqqq qq qqqqq qqqqqqq q q qqqqq q q qqqq q q qq qqqq qqqqqqq qqqqqq q qq qqqq q q q qqqq q q qqqqqq qqqq q qq qqqqqqq q q q qqq q q qqqqqq q qq q qqqqq q qqq qqq q qqqqq q qqq q qqqqq qqqqqq qqqqq q qqq qqq q q q qqq q qqqqqqq q qqqq q qq qqqqqq q qqqq qqq qq qqqq q q q q qqqq qqqqqqq qqqqqqq qqqqqq q qqq qqqq qqq qqqq q q q qqqq qqqqqq q q q qq qq q qq q qq q q qqqq qq q qqq q q q q qqq q qqq q q qq
qq qqq q q qqqq qq qqqqqq q qqq q q qq
qqq q qq q q
qq q qqq qqqq q qqqqq q qq qqqqq q q q q qqq qqq qqqq q q q qqqq q q qqqqq qqqqqqq q q qqqqq q q qqqq q q qq qqqq qqqqqqq qqqqqqq qq qqqq q q q qqqq q q qqqqqq qqqq q qq qqqqqqq q q q qqq q q qqqqqq q qq q qqqqq q qqq qqq q qqqqq q qqq q qqqqq qqqqqq qqqqq q qqq qqq q q q qqq qqqqqqqq qqqqq q qqqqqqqq q qqqq qqq qqq qqqq q q qqqqq qqq qqqq qqqqqqq qqqqqq q qqq qqqq qqqqqqq q q q qqqq qqqqqq q qqqq qq q qq q q q qq qqqq qqq qqq q q q q qqqq qqq q q q qqqqqq q q qqqqqq qq qqqq q qqq q q qqqqq q qq q qqqq qqq qqqq q qq qqq q qq qqqqq q q q q qqq qqq q qqq q q q qqqq q q qqqqq qqqqqqq qq qqqqq q q qqqq q qqq qqqq q qqqqqq qqqqqqq qq qqqq q q q qqqqq q qqqqqq qqqq q qq qqqqqqq q q q qqq q q qq qqqq q qqq qqqqq q qqq qqq q qqqqq q qqq q qqqqq qqqqq q qqqqq q qqq qqq qqq qqq qqqqqqqq q qqqq q qqqqq qq q q qqqq qqq qqqqqqq qqq qqqq qqq qqqq qqq qqqq q qqqqq q qqq qqqq qqqqqqqq q q qqqq q qqqqq q qqqqqq q qq q q q qq qqqqqq q qqq q qq q qqqq qqq q q qqqqqqq q q qqq qqq qqqqqqq qqq q q qqqqq q qqq qqqq qqq q qqqq qqqqq q qq qqqqq q q q q qqq qqq q qqq q q qqqqq q qqqqqq qqqqqqq qq qqqqq q q qqqq q qq qqqqq qqqqqqq qqqqqqq qqqqqq q qq qqqqq q qqqqqq qqqq q qq q q qqqqq qq q qqq q q qq qqqq q qqq qqqqq q qqq qq
99%
qq
q qqqqq q qqq q qq qqq qqqqqq qqq qq qqqq qqq qqq qqq qqqqqqqq q qqqq q qqqqq qqq q qqqq qqqqqqqqqq q qqqqqq qqq qqqq qqq qqqq q q qqqq q qqqqqq q qqqqqqq q q q qqqq qqqqqq q q qqqq q qqq q qqq q qqqqqq q qq q q q q qqqq q qqq q q qqqq qqq q qq qq q q qq qqqqqq qqq q q qqqqq q qq qqqqq qqq q qqqq qqqqqq q qqqqqq qq q q qqq qqq q qqq q qqqqqqq q qqqqq qqqqqqq q q qqqqq q q qqqq q qqqq
q
qq q qqqqqq qq qqqq qq
q qqqq q qqqqqqq q q qqqqq qqqq q qqq q qqqqqqq qqqqq
q q qqqqq q qqq qqqqq q qqq qq
q
q qqqqq q qqq q qq qqq qqqqqq qqq qq qqqq qqq qqq qqq qqqqqqqq q qqqq q qqqqq qqq q qqqq qqqqqqqqqq qq
q
qqq
q
qqq qqqq qqq qqqq q q qqq
q q qqqqqq q qqqqqqq qq q qqqq qqqqqq q q qqqqq qqq q qqq q qqqqqq qqqq q q q qqqq qqq
q
q q qqqq qqq q
qq qq q q qq qqqqqq qqq q q qqqqq q qq qqqq q qqq q qqqq qqqqqq qqqqqqq qq q qqqq qqq q qqq q qqqqqqq
q qqqqq qqqqqqq q q qqqqq q q qqqq q qqqqqqq q qqqqqq qq qqqq qq q
qqqq q qqqqqqq q q qqqqqqqqq q qqq q qqqqqq
q
qqqqq q q q qqqq q qqq qqqqq q qqq qqq q qqqqq q qqq q qq qqq qqqqqq qq
q
qq qqqq qqq qq q qqq
q
qqqqqqq q qqqq q qqqqq qqq q qqqq qqqqqqqqq q qqqqqq
q
qqq qqqq qq
q
qqqq q q qqqq q qqqqqq q qqqqqqq qq q
q
qq
q
qqqqqq q q qqqqq qqq
q
qqq q
qqqqqq qqqq
q q q qq
qq
q
qq
q
q q qqqq q
qq q qq qq q q qq qqqqqq qqq q q qqqqq q qq qqqq q qqq q qqqq qqqqqq
q
qqqqqq
q
q q qqq q qqq q
qqq q qqqqqq
q
q qqqq q qq
q
qqqq q q qqqq q q
qqqqq q qqqqqqq q qqqqq qqqqqqq qq q qqqq q qqqqqqq q qqqqqqqqqq q qqq q qqqqqqq qqqqq q q q qqq
q
q qqq qqqqq q qqq qq q q qqqqq q qqq q qq qqq qqqqqq qqq qq qq
q
q qqq qq q qqq
q
qqqqqqq q qqqq q qqqqq qqq q qqqq qqqqq
q
qqq q qqqqq
q
q
qqq qqqq qqq qqqq q q qqqq q qqqqqq q qqqqqqqqq q
q
qq q qqqqqq q qqqq
q
q qqq q qqq q qqqqqq qqqqqq q qqqqq
qq q
q q qqqq qqq q
qq q q q q qq qqqqqq qqq q q qqqqq q qq qqqq q qqq q qqq
q qqqqq q qqqqqq q
qq q qqq q qqq q
qqq q qqqq
qq
q
q qq
qq q qq
q
qqqq qq qqqq q q qqqqq q qq qqqqq q qqqqq qqqqqqq qq qqqqq q qqqqqqq q qqqqqqqqqqq q qq q qqqqq
qq qqqq q
q q q qqqq q qqq qqqq q q qqq qq q q qqqqq q qqq q qq qqq qqqqqq qqq qq qqqq qqq qq q qqq qqqqqqqq q qqqq q qqqqq qqq q qqq q qqqqqqqqq qqqqqqq q qqq qqq q qqq qqqq q q qqqq q qqqqqq q qqqqqqqqq q qqq q qqqqq
q
q qqqq
qq qqq q qqq q qqqqqq qqqqq
qq qqqq
qq
q q q q qqqq qqq q
q
q q q q q qq qqqqqq qqq q q qqqqq q qq qqq
q
q qqq qqqqq qqq
qq
q qqqqqq qqq q qq
q
q qqq q qqq q qqqqq q
q
q qqqq q qqqqqq q qqqqqq q q qqqqq q qq qqqqq q qqqqq qqqqqqq qq qqqqq q qqqqqqq q qqqqq qqqqqq q qq q qqqqqqq qqqq q q q q qqq q q qqq qqqq q q qqq qq q q qqqqq
q qqq q qq qqq qqqqqq qqqqq qqqq qqq qq q qqq q qqqqqqq q qqqq q q qqqq qqq q q
qq q qqqqqqqqq q
qqqqqq q qqq qqq q qqq qqq q q q qqqq q qqqqqq q qqqqqq qqqqq
q
q q qqqqq
q
q qqqqqq qqq q q
qq q
qqqqq
q
qqqqq
q
q qqqqqqq
q
q q qqq
q
qqq q q
q q
q qq qqqqqqqq qqq q q qqqq
q
q qq qqqq q qqq qqq qq qqqq
q
q qqqqqq qqq q qq
q
q qq
q
qqqq q qqqqq qqq qqqq q qqqqqq qqq qqqq q q qqqqq q qq qqqq q q qqqqq qqqqqqq qq q
qqqq q
q
qqqq
q q q qqqqq qqqqqq q qq q qqqq qqq qqqq q q q q qqq q q qqqqqqq q q qqq qq q q qqqqq q
q
qq
q
qq qq q
q
qqq
q
q qqqqq qqqq qqq
q
q q qqq q
q
qqqqqq q
q
qq
q
q q qqqq q
qq
q
q
qq qqq
q
qqqqqq
qqqqqqq q qqq qqq q qqqqqq
q q q
q
qq
q
q qqqqqq
q qqqqqq q
q
qqqq
q
q qqqq
q
q
q qqqq
qq
qqq q q
qq q
qqqqq
q
qqqqq
qq
qqqqqqq q q q qqq
q
qqq q q
qq
q qq qqqq qqqq qqq q q qqqqq
q qq qqq
q
q qqq qqq qq qqqqq q qqqqqq qqq q qqq q qqq qqqq
q
q
qqqq
qqq qqqq
q qqqqqq q
qq
q
q
qq
q
q q
qqqq
q qq qqqq q
q
q
q
q
qq
qqqqq
qq q
q
q
q qqq
q
q
q
q
q
qq q q
q
q
qqq q
q
qqqq q qq q
qq
qq qq q qqqq q q q q q
qq q q qqqqqq
q q q qqq qq q q
q
qqqq
q
qqq
q
qq qq q
q
qqq
q
q qqqqq qqqq qqq
qq q qqq q
q
qqqqqq
q
q
qqqqq qqqq qqq
q qqq qqq
q
qqqqqq
qqqqqqq q qqq qqq q qqqqq
q q q q
q
qq
q
q qqqqqq q qqqqq
q qqq qqq
q
qqqqqqq
qqqqq
qq
qqq q qqq q
qqqqqq qqqqqqq qqqqqqq q q q qqqq
qqq q qqq q qq
qqqq qqqqqqq q q qqqqq
q qq qqq
q
q qqq qqq qq qqqqq
q qqqqq
q
qqq q qqq q qqq qqqq q qqqq
q qqq qqqq
q qqqqqq qqq qq
qq
q
q
qqqqq
q q
q q
qq
q q
q qq
q
qq
q
qqqqqq qq
q
q qqq q qqq
q
q
q q q
q
q
qqq qqq
qqq q qq q
q
q
qq qq q qqqq q q q qq
qq qq qqqqqqq q q qqq qq q q qqqqq qqqq qqq qq q
q
qqq
q
q qqqqq qqq q qqq qq q qqq q qqqqqqq q qqqqqq qqqq qqq
q
qqq qqq qqqqqqq qqqqqqq q qqq qqq q qqqqqq q q qqqqq qqqqqqq qqqqqqq qqq qqq
q
qqqqqqq
qqqqqqq qqq q qqq q
q
qqqqq qqq qqqq qqqq qqq q q q qqqq
q
q q q qq q q qq
qqqq qqqqqqq q q
q
qqqq q
q
q qqqq qqqq qqq qq qqqqq q qqqqqq qqq q qqq q qqq qqqq q qqqqq qqq qqqq q q
qqqqq qq q qq
qq q q qqqqq
q qq qqq
q q qqq
q
qq qqqqqqq
q
q
qq q
q
q q
qqq
q
q
q
q q qqq
qq qqqqq q q qq q qqqq qq q qq
qq q q q qqqq qq q qqqqqq qq qqq qq q q qqqqq qqqq
q
qq qq q qqq
q
q
q qqqqq qqq q qqq qq qqqq q qqqqqqq q qqq qqq qqqq qqq
q
qqq qqq
q
qqqqqq
q
qqqqqq q qqq qqq qqqqqqq q q
q
q
qqq qqqqqqq
q
qqqqqq q
q
q qq
q
q
q
qqqqqq q qqqqqq qq q q qqq q qqqqqq qqq q qqq qqqq qqq q q q q
qqq qq q q qq q q qq
q
qq
q qqqqqqq q
q
q
qqqq q qq qqqq qqq q
qqq q
q qqqqq q qqqqqq qq q q qqq q qqq qqqq
q
qqqqq qq q qqqq q
q
qqqqq qq q qqqq q q qqqqq
q
qq qqqq qqqq
q
qq qqqqqqq
qq
qq q
q
q q
qqqqqq
q q q
qq
qq qqqqq qq q
q
q qqqq qq q qqqq q q qqqqq qq q qqqq qq
q
qqqq qq q q qqqqq
q
q qq
q
qq
q
q q qq
q
q
q
q qqqqq qqq q qqq qq qqqq q qqq qqqq qqqq qqq qq qq qqq
qq
qq qqq qqqq
q
qq
q
qqqqqq q qqq qqq qqqqqqq q q qqqqq qqqqqqq qqqqqqq
q
q q qqqq
q
qqqqqq q
q
qqqqq qq q q
q
qq q
q
qqqqq qqq q qq q qqqq qqq q q q
q
qqq qq q q qq q q qq
q
qq
qqqq
q
qq q q
q
q
qqqq
q
qq qqqq qqq q
qqq
q
q qqqqq qq qqqqq qq q q qqq
qq
qq qqq q qqqqq
q qq q qqqq q qqq qqq qq q qqqq q q
q
q
qqq
qqqqqqq qqqq qqq
qqqqqqq
q
q
q
q qq
q
qqq
q
q
q
q
q q q
q
q
qq qqqqq qq q qqqqqq qq q qqqq q q qqqqq qq q q qqq qq
q
qqqq qq q q qqqqq qq qq qqq qq q qq
q
q
q
q qqqqq qqq q qqq qq qqqq q qqq qqqq qqqq qqq qq qqqqq
q
qqq qqq qqqqqqq
q
qq
q
qq
q q qqqqqq qqqqqqq q q qqqqq qqqqqqq qqqqqqq q q q qqqq
q
qqqqqq q qqqqqq qq q
q
qqq q
q
qqqqq qqq q qq q qqqq qqq q q
q
qqqq q
q q q qq q q qq
q
qq
qq
q
q qqq q q q
q
qqq q
q
qq qq
q
q
qqq q q
qq q
q qqq
q
q qq
q
qqqq qq q q qqq q qqq
q
qq q
qq
q q
q
q qq q qqqq q qqq qqq qq q
q
qqq q q q
qqqq qqqqqqq qqqq qqq qqqqqqq q
q
qqqqq qq q
qqqq
q q qqqqq qqqqq qq q qqqqqq qq q qqqq q q qqqqq qqq q qqq qq
qq
qqq qq q qqqqqq qq qq qqq qq q qqqq qqq qqqq qqq qqqq qq qqqq q qqq qqq q q qqq qqq qq qqqqq qqqq qqq qqqqqqq
q
qqqqqq q qqqqqq qqqqqqq q q qqqqq qqqqqqq qqq qqqq
q q q qqqq
q
qqqqqq
q
q
qqqq q qq q q q
q q q
q
q
q
qqq qq q q qq q qqqq qqq q q q qqqq
q
q q q qq q q qq
qq
q
q
q
qq
q
qq
q q
q
q
qqq q
q
qq qqqq qqq q
q
qq qq qqqqq qq qqqqq q q q q qqq
q q
qq qqq
q q
q
q q
q
q qq q q
q
qq
q qqq qqq qq q
q
qqq q q qqq
qq
q
qqqqqq
qqqq
q
qq qqqqqqq qq qqqqq qq
q
qqqq q qqqqqq qqqqq qq q qqqqqq qq q qqqq q qqqqqq qqq q qqq qq
q
qqqq qq q qqqqqq qq qq qqq qq q qqqq qqq qqqq qqq qqqq qq qqqqq qqq qqq q q qqq
q
qq qq qqqqq qqqq qqq qqqqqqq q qqqqqq q qqqqqq qqqqqqq
q q qqqqq qqqqqqq qqq qqqq q q q qqqq qqqqqqq q
q
qqqq q qq q
q
q
q q q
q
qq
qqq qq q q qq q qqqq qqq q q qq
qqq
q
q q q q
q q q
q
q
q
q
qq
q
qq q
qq
q
q
q
q
qqq q
q
qq qqq
q qqq q q
qq
qq qqqqq q
q
qqq
qq q q q
q
qq
q q qq q qq
q q
q
qq
q
q
q qq q qqqq q qqq qqq q q q
q
q
qq q q
q
qqqq
q
qqqqqq qqqq
q
qq q qqqqqq qq qqqqq qq q
qqqq q qqqqqq qqqq q qq q
qqqqqq qqq q qq q qqqqqqq qqq q qqq qq
q
qqqq qq q qqqqqq qq qq qqq qq q qqqq qqq qqqq qqq qqqq qq qq qqq qqq qqq q q qqq qqq qq qq qq q qqqqqqq qqqqqqq q qq qqqq q qqqqqq qqqqqqq q q qqqqq qqqqqqq qqqqqqq
q
q
q
qqqq
q
qqqqqq q q qqqq q qq
q q
qq q q
q
qq
qqq qq q q qq q qqq q
q
qq q q qq
qq
q
qq q
q
q
q q
q qq
q
q
qq
q
qq qqq q
q q
q
qqq q
q
qq q
qq
q
q
qq q q
q
q
qq qqqqq
q
q
qq
q
qq q q q
q
q
qq
q
qq q
q
q
q q q
q
q q
qq qq q qqqq q qqq qqq q q q
q
q
qq q q qq
q
qq qqqqqqq qqqq qqq
q q qqqq q qqqqqqq qq qqqqq
q qqqqqq qqqq q qq q
q
qqqqq qqq q qq q qqqqqqq qqq q qqq qq
qqqqq qq q qqqqq q qq qq qqq qq q qqqq qqq qqqq qqq qqqq qq qq qqqqqq qqq q q qqq qqq qq qq q
q q qqqqqqq qqqqqqq q qq
q
q
qq qqqqqqq qqqqqqq
q q qqqq q
qqqqqqq qqqqqqq
q
q q q
qqq qqqqqqq q
qq
qqq q qq q q
q
q q
q
q
qq
qqq qq q q qq q qqq q qqq q q
qq
qq
q
qq q
q q
q q q q
q
q
q
qq
qq q q
qq
q q
q
q
qqq q
qq
q
qqq
q
qqq q
q
q
q
qq qqqq
q
qq
qqq
qq q
q
q
q qqq q
qq q
q
q
q q
q
q
q
q
qq qq qqqqq qqqq q
q
q q q q
q
q
qq q q qqqqq qqqqqqq qqqqqqq q q
q
qqq q q
q
qqqqq qq qqqqq q qqqqqq qqqq q qq q
qqqqqq qqq qqq q q qqqqqq qqq q qqq qq qqqqq qq q qqqqq q qq qq qqq
q
q q qqq
q
qq q qqq q qqq qqqq q q qq qqqqqq qqq q q qqq qqq qq qq qq q qqq qqqq qqqqqqq q qq
qqqq qqqqqqq q qqqqqq q qqqqq q
q
qqqqqq qqqqqqq
q q
q
q
qqq qqqqqqq
q
q
q
q
q
q
q qq q
q q
q q q
q
qq
qq q qq q q qq
q
qqq q
q
qq q
q
qq
qqqqq
q
q
q
q q q q
q q
q
q
q
q
q
q
q
qq q
q
q
qqqq
q
q
q
q
q
qqq
q
qq
q q
q
q
q
q qqqqq
q
q q q
q
qq q
q
q
q
q
qq
q qq q
q
qq
q
qq
q
q
qq q q
q
qqqq qqqq q
q
q q q q
q
q
qq q q qqqqq qqqqqqq qqqqqqq q q qqqq q qq qqqqq qq
q
qq
q
q q qq
q
qqq qqq
q
q qq q
q
qqqqq qq q qqq q q qqqqqq qqq q q
qq qq qqqqq qq
q
q
q
qqq q qq qq
q
q
q
q
q
q
q
q
q
q qq
q
qqq q qqq qqqq q q
q
q
q
qqqqq qqq q q qqq
q
qq qq
q
q qq q qqq qqqq qq qqqqq q qqqqqq qqqqqqq qqqqqqq q q qqqq q qqq
qqqq qqqqqqq
q
q
q
q
qq
q q
qqqqqq
q
q
qqqq q qq q
q q
q q q
q
qq
qq q qq q q qq
q
qqq q
q
qq q
q
q
q
qq
q
qq q q
q
q q qq
q
q
q
q
q
q
q q q
qq q
q
q
q
q
qq q
q
q q
q
q
qq q
q
q
qq
qq qqqqqqq
q
q q qqqq q q
q qq
qq
q
qq q
q
qq
q
qq
qq
qq q q qqqqq qqqq qqq q q q
q
qqq q q qqqqq qqqqqqq qqqqqqq q q qqqq q qqqqqq
q
q
q
q
q
q
qq q
q qq
q
qq qqq
q q qq q
q
q
q
qqq qq
q
q
qq q
q q
qqqqq qqq q qqq qq q
q
qq
q qq
q q
qqqq q qq qq
q
qq qq
qqq
q
q qq qqqq q qqq qqqq qq qqq qqqqqqqq q qq
qq q
q
q qq
qq
q
q
q
q
qqqqqq qq qqqqq q qqqqqq qqqqqqq qqqqqqq q q qqqq q qqqqqqq qqq qqqq
q q q
q
qqq qqqqqqq q qqqqq q qq q q q
q q
q
qq
qqq q qq q q q q qqqq q q
qq q
q
q
q
qq qqq
q q
q
q q qq
q q
q
qq qq q q
qq
q q
q
qqqq
q
q
q
q qqqq qqq qqqq q qqqqq q qq q qqqq q q q
q
qqq
q
qq q qqq q
q
qqqqq q q qqqqq qqqqqqq q q qqqqq q q qqqqq qq
q
qqqq
q
qq
q
q
qq q
q
qq
qq q
q
q
q qq
q q
q
q
q
qqqq q
qqqqqq qqqq q qq q
q
q
q
qqq qq q qqq q q qqqqqq
q qq q qqqqq qqqqqqq
q qqqqq q qqqq qq qq
q
q
qqqqqqqqqq q qqq qqqq qq qqq qqqqqqqq q qqqqq qq qqqqqq
q q
qqq qqq q
q
q qq
q q
q q
q
q
qqq
qqqq
qqq qqqqqqq q q q
qqq q qqqqqq q qqqqqqq q q q qqqq qqqqqq q q qqqq q q qq q q qq q qqqqqq q qq q q q q qqqq q qqq q q
q
q
qq
q
qq q qqq q qq q
q
q
qqqq q qqq
q q
q
qqqq q
q
q q qqqq qqq qq
q
q q qqqqq q qqqqqqq q q q qqqq
q
qq q qqq
q
q
qqq
qq q q
q
qqqq
q
qq
q
qqq q q qqqq q q q
q
qqqq
q
q
q
qqq
q
q
q
qq
q
qq qq q
qqq q
q
qq qqq q qq
q
qqqq q
qqqqqq qqqq q q
q
q qqqqqq qq q qqq q q qqqqqq q qq q qqqqq qqqqqqq q qqqqq q qqqq
q
q qqq
qqqqqqqqqqq q qqq qqqq qq qqq
qqqqqqqq q qqqqq qq qqqqqq
q
q
qqq qqq
qq
qqqq q
q
q
q
q
qq q
q
qqqqqq qqqqqqq q
q qq
qq
q q
qq
q
qq q qqqqqqq
q q
q
q
qqq qqqqqq q
q qqqqq qqq q
q
qq q
qqqqqq q qq q q q q qqqq q qqq q q
qq
qq q
qq q qqq q qq
q
q
q
qqqq q qqq q q
q
qqqq q
q
q q qqqq qqq qq
q
q q qqqqq q q
q
qqqqq q q q
q
qq
q
q
qq q
q
qq
q
q
q
qq
qq q q qqqq q
q
qqqqqq q q qqqq q q q
q
qqqq
q
qqqqq q qqqqqq q qq qqqq q
q
qq
qqq q
q
qq
qqqq q q
q
qqqq qqqq q q q
q
qqqqqq qq q qqq q q q qqqqq q qqq qqqqq q qqq qqq q qqqqq q qqqq
q
q qqq qqqqqqqqqqq q qqq qqq q qq qqq qqqqqqqq q qqqq q qq qqqqqq
q q
qqq qqq qqqqqq q
q
qqq
qq q qqqqqqq qqqqqqq q q qqqq q qqqqqq q qqqqqqq q q q qqqq qqqqqq q
q qqqqq qqq q
q
qq q
qqqqqq q qq q q q q qqqq q qqq q q
q
qqq
q
qq q qqq q
q
q
q q
q
q
qqq q qqq q q
q
qqqq q
q
q q
qqq q qqq qqq
q q qqqqq q qqqqqqq q q q qqqq qqq q qqq q
q
qqqqq q q qqqqq qqqqqqq q q qqqqq q q qqqqq qqqqqqq qqqqqqq qq qqqq
q
q
q q
qqqq q
q
qq
q
qq q q qqqqq qqqq q qq q q
q
qqqq qq q qqq q q q qqqqq q qqq qqqqq q qqq qqq q qqqqq q qqq q qq qqq qqqqqqqqqqq q qqq qqq q qq qqq qqqqqqqq q qqqq q qqqqqqqq q qqqq qqq qqqqqq q q qqqqqq qqq qqqq qqqqqqq q q qqqq q qqqqqq q qqqqqqq q q q qqqq qqqqqq q q qqqq q qqq q qqq q qqqqqq q qq q q q q qqqq q qqq q q qqqq qqq q qqq q qq q q qqqqqq qqq q
q q
qqqq q
q
q qqqq q qqq qqqq q qqqqqq qqqqqqq q q q qqqq qqq q qqq q
qqqqqq q q qqqqq qqqqqqq q q qqqqq q q qqqqq qqqqqqq qqqqqqq qq qqqq q
q
q qqqqq qqqqqqq q q qqqqq qqqq q qq q q qqqqq qq q qqqq q q qqqqq q qqq qqqqq q qqq qqq
q qqqqq q qqq q qq qqq qqqqqqqqqqq qqqq qqq qqq qqq qqqqqqqq q qqqq q qqqqq qqq q q
qqq qqq qqqqqqq q qqqqqq qqq qqqq qqqqqqq q q qqqq q qqqqqq q qqqqqqq q q q qqqq qqqqqq q q qqqq q qqq q qqq q qqqqqq q qq q q q q qqqq q qqq q q qqqq qqq q qqqq q q q q qqqqqq qqq q q
q
qqqq q qq qqqq q qqq q qqqq qqqqqq q qqqqqq q q q qqqq qqq q qqq q qqqqqq q q qqqqq qqqqqqq q q qqqqq q q qqqq q qqqqqqq q qqqqqq qqqqqq qq
q q qqqq qqqqqqq q qqqqqq qqqq q qqq q qqqqq qq
qqqqq q q qqqqq q qqq qqqqq q qqq qq
99%
DA
RT
−100
0
100
200
300
400
−100
0
100
200
300
400
50 75 100 125 150
Generation (GW)
LMP($/MWh)
LMP versus Generation, PJM, 2012
Supply curves are approximately linear, and steeper in RT

Two-Settlement Market Model: Nomenclature
ﬁnancial physical
demand
supply
DA LMP ˆp(ˆx)
v+ + ˆx = v− + ˆy
INC v+ DA gen ˆx
DEC v− DA load ˆy
RT LMP p(x)
x = y
RT gen x
RT load y
Observations from the market data
ˆx ≈ x, ˆy ≈ y, ˆp ≈ 0, p ≈ 0, p > ˆp
Next we show how p depends on ˆx

RT Supply Curve Conditioned on DA Generation
ˆx
ˆp
DA
x
p
RT
ˆx
Inﬂexible Flexible
RT supply curve steeper: dispatchable generation
Intercept of RT supply curve depends on ˆx

Modeling DA/RT Supply Curves
ˆx, x
ˆp, p
ˆp(ˆx) = âˆx
ηˆx
p(x) = ax + b
ˆx
Inflexible generation uniformly distributed (proportion η)
Representing b and η in terms of â and a
p(ηˆx) = 0
p(ˆx) = ˆp(ˆx)
=⇒
b = (â − a)ˆx
η = 1 − (â/a)

Generation Cost Depends on DA Generation
x ˆx, x
ˆp, p
ˆp(ˆx) = âˆx
ηˆx
p(x) = ax + b
ˆx
Generation cost = + = DA gen cost + flexible gen cost
c(ˆx) = η
ˆx
0
âzdz +
x
ηˆx
(az + b)dz
= η
ˆx
0
âzdz +
x−ηˆx
0
azdz

Cost Min. ⇐⇒ Dispatch Conv. ⇐⇒ Price Conv.
x
p > ˆp
if ˆx < x
p = ˆp if ˆx = x
ˆx, x
ˆp, p
ˆp(ˆx) = ˆaˆx
ηˆx
p(x) = ax + b
ˆx
x > ˆx: cost of generation dispatched in RT =
Generation cost = +
p > ˆp
If we had ˆx = x, cost would be minimized: savings = , p = ˆp

Cost Min. ⇐⇒ Dispatch Conv. ⇐⇒ Price Conv.
x
p < ˆp if ˆx > x
p = ˆp
if ˆx = x
ˆx, x
ˆp, p
ˆp(ˆx) = ˆaˆx
ηˆx
p(x) = ax + b
ˆx
x < ˆx: cost reduction of generation descheduled in RT =
Generation cost = +
p < ˆp
If we had ˆx = x, cost would be minimized: savings = , p = ˆp

Two-Settlement Market Model: Complete Specification
DA supply curve: ˆp(ˆx) = âˆx
RT supply curve: p(x) = ax + b
RT load L is fixed: x = L
â and a are fixed with â < a, and so η = 1 − (â/a)
b is subject to an indepedent disturbance δ
b = ¯b + δ = (â − a)ˆx + δ
Empirical estimation
Year
ˆp = âˆx p = ax + b
η
â (×10−3) R2 a (×10−3) b R2
2012 0.37 0.93 0.68 −27.5 0.30 45.4%
2013 0.42 0.91 0.76 −31.8 0.34 44.6%
2014 0.56 0.55 1.72 −108.2 0.18 67.3%
2015 0.41 0.78 0.86 −44.7 0.28 53.0%

Two-Settlement Market Model: Complete Specification
L0: market’s forecast about L, the RT load
δ0: market’s forecast about δ, the RT supply function disturbance
Market minimizes the forecast generation cost over ˆx
min
ˆx
ˆc(ˆx) = η
ˆx
0
âzdz +
L0
ηˆx
(az + (â − a)ˆx + δ0)dz
DA generation
ˆx0 = L0 + (δ0/a)
Spread measures the forecast accuracy
s0 = ˆp0 − p0 = a(L0 − L) + (δ0 − δ)
Financial efficiency and economic efficiency are aligned
price convergence ⇐⇒ cost minimization
δ=0
⇐⇒ dispatch convergence

Theory of Virtual Bidding
While virtual bids do not affect RT generation, they affect DA
generation and therefore DA LMP, RT LMP and generation cost
We formulate a game with virtual bidders as strategic players, based
on the two-settlement market model
N: number of virtual bidders
Li : virtual bidder i’s forecast about L
δi : virtual bidder i’s forecast about δ
vi : virtual bidder i’s quantity bid (INC if vi > 0, DEC if vi < 0)
DA generation: ˆx = ˆx0 − i vi
DA LMP: ˆp = âˆx
Virtual bidder i’s forecast RT LMP: pi = aLi + ¯b + δi
Virtual bidder i’s forecast profit: πi = (ˆp − pi )vi

Solving the simultaneous FOCs
∂πi (vi , v−i )
∂vi
= 0, i = 1, . . . , N
yields
i
vi =
N(aL0 + δ0) − (a i Li + δi )
(N + 1)a
Equilibrium spread measures the average forecast accuracy
s∗
=
a(L0 − L) + (δ0 − δ) + i (a(Li − L) + (δi − δ))
N + 1
=
s0 + i (a(Li − L) + (δi − δ))
N + 1

Suﬃcient Condition of Price Convergence
If
1
N
i
(a(Li − L) + (δi − δ)) < a(L0 − L) + (δ0 − δ),
then
|s∗
| < |s0|
Cournot Theorem for Virtual Bidding
If
1
N
i
(a(Li − L) + (δi − δ)) → 0 as N → ∞,
then
|s∗
| → 0 as N → ∞

Profitability and Price Convergence
Virtual bidder i makes a positive profit if and only if its participation
drives the spread toward zero:
s∗
vi > 0 ⇐⇒
0 < s∗ < s∗
−i , s∗
−i > 0
s∗
−i < s∗ < 0, s∗
−i < 0
,
where s∗
−i is the equilibrium spread without the participation of i
Screening out unqualified virtual bidders with poor forecast accuracy
Introducing more qualified virtual bidders into the market
The virtual bidding mechanism is self-incentivizing: a virtual bidder
can make more profit by improving its forecast accuracy, which is also
favorable to the market

Proﬁt and Cost Savings of Virtual Bids
Market’s DA generation: ˆx0 = L0 + (δ0/a)
Optimal DA generation: ˆx∗ = L + (δ/a)
Net INC of the virtual bids: v
v∗ = ˆx0 − ˆx∗ minimizes the generation cost and induces zero spread
Proﬁt of the virtual bids
f (v) = −av2
+ av∗
v
Cost savings of the virtual bids
g(v) = −(aη/2)v2
+ aηv∗
v

Profit and Cost Savings of Virtual Bids
v
f(v)
g(v)
v∗
2v∗
η = 0.25
v
f(v)
g(v)
v∗
2v∗
η = 0.5
v
f(v)
g(v)
v∗
2v∗
η = 0.75
profit > cost savings profit < cost savings
The generation cost savings may not recover the profit of the virtual
bids when v is small
As η increases, the cost savings are more likely to recover the profit
The more competitive the virtual bidders, the more likely that the
cost savings recover the profit

Eﬀectiveness of Virtual Bidding: Empirical Estimation
q
q
q
q
−5.0
−2.5
0.0
2.5
5.0
2012 2013 2014 2015
Year
Spread($/MWh)
Virtual Bids
q No (Estimate)
Yes (Market Data)
Mean of Spread with and without Virtual Bids, PJM
Year
Cost (Billions) Savings (pct. Proﬁt (pct.
No VB VB of No VB Cost) of Savings)
2012 22.14 21.71 1.95% 8.03%
2013 24.03 23.66 1.53% 2.86%
2014 29.58 28.48 3.73% 1.59%
2015 20.29 20.24 0.23% 60.21%

Conclusion
A successful fusion of data and theory: market data → market model
→ theory and implications → methodology of estimation → empirical
evidence by market data
We propose a two-settlement market model which explains various
phenomena in the market, and provides a methodology of estimating
the generation cost
The proposed model aligns financial efficiency and economic
efficiency, which serves as the basis of the theory of virtual bidding
Virtual bidding has improved the market efficiency: (i) comparative
analysis before and after virtual bidding in CAISO; (ii) estimation of
cost savings and price convergence by virtual bids in PJM
There still exist substantial profitable opportunities: (i) market virtual
bidders make profits; (ii) simple machine learning algorithms can be
profitable on top of the market virtual bids

All of Virtual Bidding: A Data-Driven Approach

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (6)

Semelhante a All of Virtual Bidding: A Data-Driven Approach

Semelhante a All of Virtual Bidding: A Data-Driven Approach (15)

Último

Último (20)

All of Virtual Bidding: A Data-Driven Approach