© 2014 Deloitte MCS Limited. All rights reserved.
Testing and
validating stochastic
economic scenarios
Shaun Lazzari
29 May 2014

© 2014 Deloitte MCS Limited. All rights reserved.
Agenda
• Using ESGs
‒ Purpose
‒ Process (providers, groups and business units)
‒ Solvency II requirements
• Formulating calibration assumptions
‒ Required assumptions
‒ Data challenges
‒ Potential solutions
• Validating scenario sets
‒ Aims
‒ Analyses
• Future challenges
2

© 2014 Deloitte MCS Limited. All rights reserved.3
Using ESGs

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
What do ESGs do?
• Generate many scenarios for future economic variables
• Asset classes:
‒ Nominal rates
‒ Real rates
‒ Inflation
‒ Equities
‒ Property
‒ Credit spreads / default probabilities
‒ Alternatives
‒ Exchange rates
4
0 5 10 15 20 25 30 35
Time (y)

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
Purposes
• Two key types of ESG model:
• Application: Monte Carlo approach especially useful for valuation when liabilities
involve non-linear cashflows:
‒ Options/guarantees
‒ Path-dependence
‒ Management actions
5
• Market-consistent valuation (for reporting)
• HedgingRisk neutral
• Risk/return quantification
• Regulatory/economic capital calculation
• Investment strategy setting
• Pricing
Real world
(Deflator-based models incorporate features of both types of model)

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
Stochastic modelling for valuation
Liability values found as expected value of discounted projected cashflows:
• Risk-neutral means no arbitrage opportunities:
‒ Expected PV of any investment strategy is equal to amount invested today
‒ In contrast to real-world simulation, where risk premiums may be used
6
ESG
Discount
factors
Other
outputs
Liability
info
Projected
cashflows
(Market)
value
average

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
Market consistency
• Risk-neutral ESG models are calibrated to market data
• Calculated values of liabilities (which are complex financial contracts) can be
thought of as being a “market price”7
Calibration
targets (market
prices)
Calibration
Model
parameters
Scenario
generation
Economic
scenarios
Pricing
Scenario-
calculated
prices
ESG

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
Provision of scenarios: a typical process
8
ESG
provider
Group
office
Business
unit
• Software
• Calibrations
• Assumptions/insights
• Scenario files
• Software?
• Calibrations?
• Scenario files
• Business-unit specific
assumptions and file
requirements
• Company-specific
assumptions and file
requirements
Data
Data
Data

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
Provision of scenarios - challenges
• For example:
‒ Ownership of assumptions
‒ Adequate validation/challenge of assumptions
‒ Meeting ad-hoc requirements
• Often Business Units do not have access to software / provider contact
themselves, perhaps due to:
‒ Cost
‒ Resource/expertise requirements
• We are seeing reliance on third party providers and/or group centralisation
increasing over time
‒ For software and resources… but not assumptions!
• For CEE calibrations (e.g. Czech Koruna), lack of market data can make
calibration difficult
9

© 2014 Deloitte MCS Limited. All rights reserved.
Using ESGs
Solvency II
Article 126
“The use of a model or data obtained from a third-party shall not be considered to be a
justification for exemption from any of the requirements for the internal model set out in
Articles 120 to 125.”
• Use test
• Statistical quality standards
• Calibration standards
• Profit & loss attribution
• Validation
• Documentation
As an ESG provider, we find we get many more questions and challenges now
than we used to – this is good!
10

© 2014 Deloitte MCS Limited. All rights reserved.11
Formulating calibration assumptions

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Required assumptions
• Aim wherever possible to calibrate to today’s market price data
• Projected behaviour based upon these prices:
12
3
6
9
15
30
0%
20%
40%
60%
1
3
5
7
9
12
20
30
Option term
(y)Swap term (y)
Market implied vol (ATM)
40%-60%
20%-40%
0%-20%
Drift
• Linear payoffs – bonds, forward
contracts
Volatility, skew, autocorrelation etc.
• Non-linear payoffs - options +
other derivatives
0%
1%
2%
3%
4%
0 10 20 30 40
SpotRate
Term (y)
Initial yield curve

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Required assumptions
• Ideally:
+ inter-asset class correlation assumptions!
13
Nominal rates
• Initial yield curve
• Swaption
prices/implied
vols
Real
rates/inflation
• Initial yield curve
• Volatility & mean
reversion levels
Equities &
other indices
• Option
prices/implied
vols
• Forward
dividends
• Dividend
volatility & mean
reversion
Credit
• Initial credit
spread curves
• Spread volatility
FX
• FX option
prices/implied
vols

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Data challenges
Ideally, we would calibrate using targets solely sourced from market prices. In
practice, many reasons why not possible:
14
• Swaption prices based on swap rates – inconsistency if
using government curveNominal rates
• Few economies issue inflation-linked bonds
• Derivatives on these bonds are even rarerReal rates
• Insurers generally interested in long term implied
volatilities – very scarce data
• For property etc., no liquid derivative markets
Equities & other indices
• Data very fragmented as multiple issuers – some indices do
exist for major economies
• Few derivatives
Credit
• Few liquid cross-asset class derivativesCorrelations

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Important features of an assumption-setting approach
The issues described have long existed and many workarounds can be used. In
a Solvency II world, these must be well-justified!
• Informed by relevant data
• Limited and well-validated use of expert judgement
• Stability over time
15

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Solutions
1 – Use of historic data
• Common approach for several targets
‒ Volatility – property, inflation, credit…
‒ Correlations
• Note implied volatility ≠ volatility
‒ Bias
‒ Observed volatility says nothing about forward-looking term structure, skew
16
0
20
40
60
80
1990 1993 1995 1998 2001 2004 2006 2009 2012
IV/vhistoricvol(%)
Time
VIX implied
60D vol
Average 3.5%
difference

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Solutions
2 – Use of proxy data series
• Asset class may be approximated by a related, more established class for which
data exists
• Substitute assumption should be well-validated:
‒ Statistically
‒ Analysis of underlying
drivers
• May seek to make appropriate adjustments to proxy data
17
1998 2001 2004 2007 2009 2012
CECE CTX

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Solutions
3 – Third party guidance
• Calibration assumptions are, ultimately, prices of simple financial contracts
‒ Request quotes from banks – they are the market makers!
‒ Seek assistance from data provider
‒ Inspect regulatory returns
• With Solvency II, insurer still required to take ownership of assumptions
18

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Example – Czech/CEE equity
• Only short-term options traded for CECE
‒ Would like a full surface
• Could we use a major EUR index like the Eurostoxx or DAX as a proxy?
Historic behaviour:
19
1998 2001 2004 2007 2009 2012
CECE
Eurostoxx 50
DAX

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Example – Czech/CEE equity
Historic volatility:
Short term ATM implied volatilities:
20
CECE Eurostoxx 50 DAX
27.2% 23.1% 24.44%
0
5
10
15
20
25
30
May-14 Oct-15 Feb-17 Jun-18
ATMIV(%)
Expiry date
CECE
DAX
Eurostoxx 50
Unusual downward slope!

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Example – Czech/CEE equity
Higher moments:
21
Density
Historic log returns (annualised)
CECE
Eurostoxx 50
CECE Eurostoxx 50
Skewness -0.7 -0.8
Kurtosis 6.2 5.9

© 2014 Deloitte MCS Limited. All rights reserved.
Formulating calibration assumptions
Example – Czech/CEE equity
• Lot of choice as to how incorporate these observations into assumptions
‒ But this analysis provides us with evidence to back-up approach
• Approach should be robust – i.e. stable over time
22
Calibration
assumptions
Short-term
market data
Difference
in historical
volatility
Similar
higher
moments,
hence skew

© 2014 Deloitte MCS Limited. All rights reserved.23
Validating scenario sets

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Aims
• Having made a set of scenarios, must adequately validate them
• Seek to verify:
• Ideally in as automated and judgement-free way as possible
24
Arbitrage-
freeness
Market
consistency / fit
to calibration
targets
Model stability
Compatibility
with cashflow
model

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – no arbitrage/leakage
Test both raw outputs and more complex (dynamic?) strategies:
• Means of quantifying error:
‒ Maximal error
‒ Confidence intervals
‒ Terminal leakage
25
0.8
0.9
1
1.1
1.2
0 10 20 30 40
Time (y)
Govt. CMI Term 15
0.8
0.9
1
1.1
1.2
0 10 20 30 40
Time (y)
Equity TRI
Undervaluation of guarantees

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – market consistency
Compare market prices against those found through pricing using scenarios:
26

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – market consistency
Compare market prices against those found through pricing using scenarios:
27
0%
1%
2%
3%
4%
5%
0 5 10 15 20 25 30 35 40
Spotrate
Term (y)
Average deflator
Yield curve
95% confidence interval

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – market consistency
• Monte Carlo prices are an average
→ can use similar pass/fail criteria used for no-arbitrage tests
• Can break down error into two parts:
• Significance of sampling error best quantified through comparing prices, not vols
etc.
28
Fitting error
Sampling
error
Market
consistency
error
Targets - Fitted Fitted - Generated

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – convergence
• Are we convinced enough simulations have been used?
• In more volatile environments, more scenarios required
29
10%
15%
20%
25%
0 1,000 2,000 3,000 4,000 5,000
10x10swaptionImpliedvol
# simulations
Simulations
Model

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – out of sample testing
• Wish to verify model is not over-fitted, but instead has some predictive power
‒ If it doesn’t, ESG is pointless!
Do not always have excess data available, but sometimes we do!
30
10%
15%
20%
0 5 10 15
Impliedvolatility
Term (y)
Simulated Market Fitting
o Bond prices
o Swaption cube points
o Interest rate caps
o Intermediate points on
implied vol term structure

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – distributional features
• Out-of-sample contracts most likely to be mispriced if output distributions are “not
sensible”
• Extreme distributions may also impact ALM model compatibility
• Additionally, consider changes in distributional statistics over time – are these
consistent with changes in calibration assumptions?
31
-80% -60% -40% -20% 0% 20% 40% 60% 80%
Log-return
SVJD
Black-Scholes

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – distributional features
• Out-of-sample contracts most likely to be mispriced if output distributions are “not
sensible”
• Extreme distributions may also impact ALM model compatibility
• Additionally, consider changes in distributional statistics over time – are these
consistent with changes in calibration assumptions?
• Aside: have seen other European regulators asking firms to test multiple models
32
-4%
0%
4%
8%
12%
16%
20%
24%
28%
0 5 10 15 20 25 30 35 40
Spotrate
Time (y)
Hull-White
-4%
0%
4%
8%
12%
16%
20%
24%
28%
0 5 10 15 20 25 30 35 40
Spotrate
Time (y)
LMM
-4%
0%
4%
8%
12%
16%
20%
24%
28%
0 5 10 15 20 25 30 35 40
Spotrate
Time (y)
LMM-DDSV

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Analyses – calibration stability
• For a given model, finding optimal parameter set is a hard problem
1) Test optimisation routine
‒ Generate targets from the model
‒ Fit to these targets – should be able to achieve exact fit, and ideally same parameters as
used to generate targets
2) Test goodness-of-fit over time
‒ Fit to historic targets
‒ Asses fit in range of market conditions, and stability over time
3) Test parameter stability
‒ Make small adjustments to initial guess – should have small impact on outcome
33

© 2014 Deloitte MCS Limited. All rights reserved.
Validating scenario sets
Doing all this analysis
• Some of this is one-off work (validating optimisation routine etc.)
• Model is not particularly firm-specific – provider may be best to validate
‒ Firm need only demonstrate evidence and understanding
• If Business Unit is reliant on Group for scenarios, must seek to request sufficient
information to calibrate
‒ e.g. to accurately price swaption, many outputs required
• Much of regular validation process can be automated
34

© 2014 Deloitte MCS Limited. All rights reserved.35
Future challenges

© 2014 Deloitte MCS Limited. All rights reserved.
Future challenges
Immediate issues
• ESGs models have reached a mature stage where most calibration targets can
be achieved:
‒ Initial yield curves
‒ Option surfaces
‒ Volatility cubes
• Some advances can still be made with regards to credit modelling
• Automation an area of focus as volume of ESG file required increases
‒ Quicker delivery
‒ Sensitivities
‒ Nested stochastic etc.
36

© 2014 Deloitte MCS Limited. All rights reserved.
Future challenges
Longer term
• Emerging standards, including Solvency II and IFRS, continue to emphasize
market consistency - generally a good thing.
• Insurance definition is based on classical option pricing theory (replicating
portfolios); many assumptions:
• Limitations highlighted post-2008!
37
Forbidden
• Bid-ask spreads
• Market impact of trades
• Information asymmetries
• Taxes
• Solvency capital requirements and
costs of holding these.
• Collateral posting requirements
• Risk of default on derivatives
• Illiquidity premiums or other non-cash-
flow valuation effects
Required
• Investment and unlimited borrowing at a
single risk free rate.
• Unlimited and infinitely-divisible supply of
underlying assets.
• Continuous-time trading (24/7)
• Buying and selling with no impact on the
market price.
• Consensus on possible price moves in
the underlying asset.

© 2014 Deloitte MCS Limited. All rights reserved.
Future challenges
Longer term
• Banks have adopted adjustments to counter weaknesses in theory:
• These innovations may hit insurers first via IFRS rather than Solvency II
38
Credit valuation
adjustment
CVA
Debit valuation
adjustment
DVA
Funding valuation
adjustment
FVA
Allowance for possible default by
derivative counterparties
Reduce stated liabilities with an
allowance for own default.
Allowance for funding of derivative
position (borrowing over the risk free
rate, stock lending, collateral posting).

© 2014 Deloitte MCS Limited. All rights reserved.
Future challenges
Longer term
• Real-world modelling has itself advanced greatly in recent years due to
Solvency II
‒ Diverged from risk-neutral approach
• Incorporating these “real-world” features into market-consistent modelling will
bring these two types of modelling closer together
• Working towards a Grand Unified Model!
39

© 2014 Deloitte MCS Limited. All rights reserved.
Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited (“DTTL”), a UK private company limited by guarantee, and its network of
member firms, each of which is a legally separate and independent entity. Please see www.deloitte.co.uk/about for a detailed description of the
legal structure of DTTL and its member firms.
Deloitte MCS Limited is a subsidiary of Deloitte LLP, the United Kingdom member firm of DTTL.
This publication has been written in general terms and therefore cannot be relied on to cover specific situations; application of the principles set out
will depend upon the particular circumstances involved and we recommend that you obtain professional advice before acting or refraining from
acting on any of the contents of this publication. Deloitte MCS Limited would be pleased to advise readers on how to apply the principles set out in
this publication to their specific circumstances. Deloitte MCS Limited accepts no duty of care or liability for any loss occasioned to any person
acting or refraining from action as a result of any material in this publication.
Registered office: Hill House, 1 Little New Street, London EC4A 3TR, United Kingdom. Registered in England No 3311052.
© 2014 Deloitte MCS Limited. All rights reserved.