Let’s face complexity September 4-8, 2017

1. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Fakult¨at f¨ur Physik
Econophysics V:
Market States and the
Subtleties of Correlations
Thomas Guhr
Let’s Face Complexity, Como, 2017
Como, September 2017

2. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Outline
mysterious vanishing of correlations: Epps effect
Gaussian assumptions and correlations: copulae
identification of market states
time evolution of market states
signatures of crisis
Como, September 2017

3. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Epps Effect
Como, September 2017

4. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Measurement of correlation coefficients
stock prices: S(i)(t), i = 1, . . . , K
returns r
(i)
∆t =
S(i)(t + ∆t) − S(i)(t)
S(i)(t)
depend on the chosen return interval Jan 2008
Feb 2008
Mar 2008
Apr 2008
May 2008
Jun 2008
Jul 2008
Aug 2008
Sep 2008
Oct 2008
Nov 2008
Dec 2008
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
S(t)/S(Jan2008)
IBM
MSFT
Cij = corr(r
(i)
∆t, r
(j)
∆t) =
r
(i)
∆tr
(j)
∆t − r
(i)
∆t r
(j)
∆t
σ(i)σ(j)
, u =
1
T
T
t=1
u(t)
assume that T is long enough −→ no noise dressing
... but what is the dependence on the return interval ∆t ?
Como, September 2017

5. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Empirical results
measured correlations suppressed towards small return intervals ∆t
−→ this is the Epps effect
0 5 10 15 20 25 30 35 40
∆t [min]
0.70
0.75
0.80
0.85
0.90
0.95
1.00
corr(∆t)/corr(40min)
ensemble of 50 stock pairs (normalized to saturation value)
Como, September 2017

6. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Goal
Variety of possible reasons discussed in finance, including highly
speculative “emergence of correlations”.
Existing studies mostly aim at schematically compensating the
Epps effect.
Being physicists, we ...
look at the data — carefully,
identify statistical causes,
develop parameter free compensation,
quantify what is left for other causes.
Como, September 2017

7. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Asynchronity — formation of an overlap
underlying fictitious
time series
actual time series
γ: last trading time
overlap ∆to(t)/∆t
with synchronous information,
outside random
Como, September 2017

8. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Compensation of asynchronity
g
(i)
∆t(t) =
r
(i)
∆t(t) − r
(i)
∆t(t)
σ
(i)
∆t
corr(r
(i)
∆t, r
(j)
∆t) = g
(i)
∆t(t)g
(j)
∆t(t)
corrasync(r
(i)
∆t, r
(j)
∆t) = g
(i)
∆t(t)g
(j)
∆t(t)
∆t
∆to(t)
term–by–term compensation by multiplying with inverse overlap
Como, September 2017

9. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Tick size and return distribution
tick-Size q
discretizes prices
returns also affected
clustering
Como, September 2017

10. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Correlation coefficient for discretized data
idea: discretization r(i)(t) → ¯r(i)(t) produces random errors ϑ(i)(t)
r(i)
(t) = ¯r(i)
(t) + ϑ(i)
(t)
corrtick(r(i)
, r(j)
) ≈
cov ¯r(i), ¯r(j)
ˆσ(i)ˆσ(j)
compensation by correcting with normalization
estimation using average discretization error
Como, September 2017

11. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Combined compensation
take both, asynchronity and discretization, into account
corr(r(i)
, r(j)
) ≈
¯r(i)
¯r(j) ∆t
∆to
ˆσ(i)ˆσ(j)
no interference, undisturbed superposition
Como, September 2017

12. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Test with stochastic processes
autoregressive GARCH(1,1) known to reproduce phenomenology of
stock price time series and their distributions
0 5 10 15 20 25 30
∆t/60
0.10
0.15
0.20
0.25
0.30
0.35
0.40
corr
asynchrony correction
tick−size correction
combined correction
full, parameter free compensation
Como, September 2017

13. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Test with real data
stocks from Standard & Poor’s 500, prices between $10.01–$20.00
0 5 10 15 20 25 30
∆t [min]
0.6
0.7
0.8
0.9
1.0
corr(∆t)/corr(30min)
corrected
parameter free combined compensation
−→ rest has other causes
Como, September 2017

14. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Results
purely statistical causes have strong impact on Epps effect
identified what is left for other causes (e.g. lags, etc)
parameter free compensation
significant better precision when estimating correlations
can easily be applied
Como, September 2017

15. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Non–Gaussian Dependencies
Como, September 2017

16. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Correlation coefficient and joint probability density function
correlation coefficient reduces complex
statistical dependence to a single number
only meaningful if dependence is
multivariate Gaussian, e.g. bivariate
fX,Y (x, y) =
1
2π
√
1 − c2
exp −
1
2
x2 − 2cxy + y2
1 − c2
if not, have to retrieve better information
from full joint probability density function
fX,Y (x, y) which contains all information
−3
−2
−1
0
1
2
3
−3
−2
−1
0
1
2
3
0.05
0.10
0.15
Como, September 2017

17. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Tools and definitions
marginal distribution: fX (x) =
+∞
−∞
fX,Y (x, y)dy
comulative distribution: FX (x) =
x
−∞
fX (x )dx
u quantile: left of x = F−1
X (u) are u percent of events
joint probability: FX,Y (x, y) =
x
−∞
dx
y
−∞
dy fX,Y (x , y )
Como, September 2017

18. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Copulae
fX,Y (x, y)
fX (x), fY (y) copX,Y (u, v)
separate statistical dependencies and marginal distributions
CopX,Y (u, v) = FX,Y F−1
X (u), F−1
Y (v)
copX,Y (u, v) =
∂2
∂u∂v
CopX,Y (u, v) .
(similar to “moving frame” or “unfolding” in quantum chaos)
Como, September 2017

19. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Comparison true versus Gaussian copulae
K return time series r(i)(t), i = 1, . . . , K
calculate standard Pearson correlation coefficients Cij
for each pair (i, j)
uniquely determines bivariate Gaussian distribution
for pair (i, j)
fi,j (x, y) =
1
2π 1 − C2
ij
exp −
1
2
x2 − 2Cij xy + y2
1 − C2
ij
evaluate corresponding Gaussian copula cop
(G)
i,j (u, v)
Como, September 2017

20. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Comparison true versus Gaussian copulae — continued
analyze true copula copi,j (u, v)
calculate distance
d(u, v) =
1
K(K − 1)/2
i<j
copi,j (u, v) − cop
(G)
i,j (u, v)
Como, September 2017

21. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Empirical study
TAQ data 2007–2010, S&P 500, more than 12 billion transactions
Quantile
1
0.0
0.2
0.4
0.6
0.8
Quantile
2
0.0
0.2
0.4
0.6
0.8
0.00
0.05
0.10
0.15
0.20
0.25
∆t = 2h
Quantile
1
0.0
0.2
0.4
0.6
0.8
Quantile
2
0.0
0.2
0.4
0.6
0.8
0.0
0.1
0.2
0.3
∆t = 30min
structure of copula stable when varying return interval
bivariate Gaussian assumption drastically underestimates
simultaneous extreme events
Como, September 2017

22. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Results
standard Pearson correlation coefficient problematic for data
which are not bivariate Gaussian
copulae provide alternative by extracting statistical
dependencies independent of marginal distributions
risk of simultaneous extreme events in real data much higher
than usually assumed
Como, September 2017

23. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Identifying Market States and their Dynamics
Como, September 2017

24. Epps Effect Non–Gaussian Dependencies Market States Conclusions
States of financial markets
market is non–stationary
different states before,
during and after a crisis
market can function in
different modes
qualitative/empirical — also
quantitative ?
S&P 500
Aug 2008
Sep 2008
Oct 2008
Nov 2008
Dec 2008
Jan 2009
700
800
900
1000
1100
1200
1300
1400
points
Como, September 2017

25. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Similarity measure
correlations provide detailed information about the market
introduce distance of two correlation matrices
ζ(T)
(t1, t2) = C
(T)
ij (t1) − C
(T)
ij (t2)
ij
i, j running index of risk element or company
t1, t2 times at which the two correlation matrices calculated
T sampling time backwards
−→ distances ζ(T)(t1, t2) array or matrix in points (t1, t2)
Como, September 2017

26. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Empirical results
T = 2 months
Como, September 2017

27. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Empirical results
T = 2 months T = 1 week
Como, September 2017

28. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Identification of market states
array ζ(T)(t1, t2) reveals changes in market structures over long
time horizons
identify states by cluster analysis
ensemble of correlation matrices C
(T)
ij (t), t = t(a), . . . , t(b)
find two disjunct clusters where distance ζ(T) from average
within each cluster is smallest
repeat that for these two clusters, and so on
stop when distances within groups comparable to distances
between grop
Como, September 2017

29. Epps Effect Non–Gaussian Dependencies Market States Conclusions
States of US financial market 1992–2010
1
2
4
5
6
7
8
3
bold face numbers label
market states
if no threshold −→
division process continued
until all correlation matrices
are identified
small numbers label year
and two–months period
Como, September 2017

30. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Market states in basis of industrial sectors
overall average correlation matrix
E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
−1.0
−0.5
0.0
0.5
1.0
Como, September 2017

31. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Market states in basis of industrial sectors
overall average correlation matrix
E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
−1.0
−0.5
0.0
0.5
1.0
the eight states:
1 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
2 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
3 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
4 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
5 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
6 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
7 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
8 E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
Como, September 2017

32. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Time evolution of US market states
subsequent formation of US market states 1992–2010
market jumps between different states
old states die out −→ states have a lifetime
how does this lifetime relate to other time scales
(e.g. times between crashes) ?
Como, September 2017

33. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Time evolution of correlation coefficients distribution
time resolved analysis of distribution p(Cij ) during the 2008 crisis
surface plot black: September 2008, red: December 2008
distribution became broader before the crisis started in
October 2008, partly due to decoupling of energy sector
very narrow during the crisis with large mean value
←→ panic
Como, September 2017

34. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Stable period within 2008–2009 crisis
correlations during crisis October 15th, 2008, to April 1st, 2009
three–week stable period January 1st, 2009, to January 21st, 2009
E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U
0.00
0.15
0.30
0.45
0.60
0.75
0.90
excluding stable period
E M I CD CS H F IT C U
E
M
I
CD
CS
H
F
IT
C
U −0.15
0.00
0.15
0.30
0.45
0.60
0.75
0.90
stable period
stable period very similar to market state number 7
Como, September 2017

35. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Results
usually stock market data analyzed “as if they were
thrown on the floor”
similarity measure reveals structural changes
clear identification of market states
time evolution of states followed −→ dynamical information
changes during 2008–2009 crisis: correlation matrix
and distributions
early warning system ?
Como, September 2017

36. Epps Effect Non–Gaussian Dependencies Market States Conclusions
Conclusions
Epps effect: it helps to look at the data
copulae: the world of finance is not Gaussian
market states: they exist, have time evolution and lifetime
Como, September 2017

37. Epps Effect Non–Gaussian Dependencies Market States Conclusions
M.C. M¨unnix, R. Sch¨afer and T. Guhr,
Compensating Asynchrony Effects in the Calculation of Financial
Correlations, Physica A389 (2010) 767
Impact of the Tick–size on Financial Returns and Correlations,
Physica A389 (2010) 4828
Statistical Causes for the Epps Effect in Microstructure Noise,
Int. J. Theoretical and Applied Finance 14 (2011) 1231
M.C. M¨unnix and R. Sch¨afer,
A Copula Approach on the Dynamics of Statistical Dependencies
in the US Stock Market, Physica A (2011) 4251
M.C. M¨unnix, T. Shimada, R. Sch¨afer, F. Leyvraz, T.H. Seligman,
T. Guhr and H.E. Stanley, Identifying States of a Financial Market,
Scientific Reports 2 (2012) 644
Como, September 2017