Model Risk Management article published by Best’s Review in September 2014.

1. Technology
Models are used broadly
within insurance organiza-
tions. For example, insur-
ers use models to set premiums;
develop appropriate reserve and
capital levels as reported in finan-
cial statements; evaluate potential
divestitures and acquisitions; set
business plans; and determine the
types of assets to purchase.
As a result, misuse of models or
errors within models can have sig-
nificant adverse consequences.
Insurer models have become
more intricate as computing power
has increased, products have become
more complex, asset/liability manage-
ment more sophisticated and regu-
latory requirements more onerous.
Furthermore, for some insurers, par-
ticularly reinsurers,the availability and
quality of data might be a concern.
For innovative or small insurers,
there may not be credible expe-
rience supporting the model-
ing assumptions. Also, new types
of models are continuously being
developed and used (such as pre-
dictive models in life insurance).
These are just some of the factors
that contribute to increased model
risk within the insurance industry.
While most insurers have invested
in integrated risk management for a
long time, model risk management
has recently become an area of
emphasis for some insurers.
Leading companies have devel-
oped MRM frameworks to mitigate
model risk and to become more
comfortable with their models.
Common elements of MRM frame-
works include policies related to
model governance, development
and implementation,and validation.
Model Governance
Strong model governance is criti-
cal to effective MRM.The boards of
directors and senior management at
best-in-class companies have recog-
nized this by sponsoring, support-
ing or leading the establishment of
MRM frameworks at their compa-
nies to help foster an MRM culture.
An MRM framework is an impor-
tant component of a company’s
overall risk management approach.
As such, the steps involved in devel-
oping an MRM framework are simi-
lar to the steps used to develop an
ERM framework.
For example, a critical element of
any risk management approach is to
identify a company’s specific risks.
A company can determine model
risk exposures by defining what a
model is, creating and maintaining an
inventory of models, and categoriz-
ing the models into low, medium and
high risk with due consideration for
materiality. Also, a company would
normally set risk limits to ensure that
the degree of model risk does not
exceed a specified risk tolerance.
Other important elements of
governance related to MRM include
policies or best practices related to
model development, testing, docu-
mentation, approvals, version and
change control, access rights and
validation.
Roles and responsibilities are
another important governance
consideration. Ideally, model devel-
opment and maintenance duties
would be segregated from model
user duties. The internal audit or
risk management departments are
often involved in measuring the
effectiveness of MRM governance.
Some companies have estab-
lished MRM officers to lead the
model risk management function.
Development and Implementation
MRM begins with model devel-
opment. Ideally, models would be
created by experienced developers
and subject matter experts using
high-quality data, robust methodol-
ogy and assumptions that are sup-
ported by credible experience in a
controlled modeling environment.
Models in a controlled environ-
ment allow for adequate testing
by Dom Lebel and Sebastien Cimon Gagnon
Strong model risk management can prevent the damaging impact that model
misuse or errors have on insurers.
Building Model Trust
Contributors:
TowersWatson’s
Dom Lebel
is a director
responsible
for financial
modeling and
reporting for the Americas Life
practice;and Sebastien Cimon
Gagnon is a senior actuarial
analyst.They can be reached at
dominique.lebel@towerswatson.com
and sebastien.cimon.gagnon@
towerswatson.com
Lebel Gagnon
Key Points
▼
The Issue: Insurance companies
are using models that are more
complex than ever.
▼
The Problem: Even slight modeling
missteps can have an enormous
impact on an insurance company’s
financials, risk management or
decision-making.
▼
The Solution: Model risk manage-
ment provides a structured, objective
system for making sure they’re reliable.
Copyright © 2014 by A.M. Best Company, Inc. All Rights Reserved. Reprinted with Permission. www.ambest.com
BEST’S REVIEW
www.bestreview.com September 2014
BEST’S REVIEW • SEPTEMBER 2014 • REPRINT 1

2. Technology
and approvals before implementa-
tion. Model documentation would
be sufficiently detailed to describe
the purpose of the model and how
it will be used; data inputs to the
model; and model methodology and
assumptions, including their ratio-
nale. Documentation also would
detail the model’s compliance with
regulations or other standards, the
testing performed and known mod-
el limitations. Updates and changes
to models would be governed by a
version- and change-control process.
While some companies have
developed standardized templates
to facilitate consistent model doc-
umentation, resource constraints,
company culture or inadequate
model governance can make it a
struggle for many. Cases with inad-
equate documentation of models
(in particular, no documentation
of model limitations) have resulted
in models being used for purposes
they were not designed for.
We recommend structuring the
documentation so it is easily main-
tained when the model is updated
and revised.
Model documentation ensures
the model was adequately devel-
oped; model owners are account-
able; and model users understand
how the model operates, how
model results should be used and
any limitations.
Model Validation
The intended purpose of model
validation is regular verification that
models are performing as expected.
While essential, this process is time-
consuming and resource-intensive.As
such, priority generally is assigned to
newer models,models with the great-
est degree of model risk and models
that have not recently been reviewed.
To avoid conflicts of interest,
the validator should be objective;
knowledgeable about the product
being modeled, modeling standards
of practice, industry practices and
regulatory requirements; and not
have an interest in the outcome of
the validation. For example, model
developers or users should not be
validators.As a result, model valida-
tions are often performed by con-
sultants. Accountability will also be
encouraged if there is a sole valida-
tor per model.
Some companies have devel-
oped model validation templates
for validators to use. The translation
of relatively high-level validation
requirements to detailed and prag-
matic validation activities can be
challenging.
Regardless of whether a valida-
tion template is available,model vali-
dators generally develop a validation
test plan.All model components are
typically validated, including inputs,
calculations and output.
An MRM governance review is
also typically within the scope of a
model validation project. It focuses
on the application of MRM gover-
nance policy; modeling oversight by
senior management and the board;
establishment and adherence to
model risk limits; maintenance of the
model inventory, roles and respon-
sibilities; and various other model
controls such as access rights,change
processes and approval procedures.
The validator typically starts by
reviewing existing model documen-
tation, including support for model-
ing assumptions and testing that has
been performed to evaluate concep-
tual soundness.
It is also important for model vali-
dators to understand the purpose of
the model and how it is currently
being used, because confirming that
the model is being used as intended
is an important validation element.
This is even more critical when no
support for assumptions exists due to
lack of company experience or indus-
try data.
Next, the validator typically
checks model output against another
model that has been independently
developed to confirm that the mod-
el has been properly implemented.
Since full reconciliations of model
output between two independent
models can be time-consuming,toler-
ances often are used to minimize the
amount of work.
Validators will often perform
additional checks such as compar-
ing model output to historical, actu-
al audited results and testing output
sensitivity to assumption changes.
We have found it useful for valida-
tors to maintain a review log that
categorizes types of issues (such as
approximation,error and misuse).
The review log can also contain
an estimate of the impact of each
issue, which can be assessed by the
validator with support from model
developers or users.
Governance
The Model Risk Management Cycle
Model
validation
Model
development
and
implementation
Use
Key Model Validation Principles
1. Model design and build need to
be consistent with the model’s
intended purpose.
2. Ensure that model validation is
an independent process.
3. Establish an owner of model
validation.
4. Ensure appropriateness of
established model governance.
5. Make model validation efforts
proportional to evidenced areas
of materiality and complexity.
6. Validate the model components.
7. Address limitations of model
validation.
8. Document the model validation.
Source: The North American CRO Council
2 BEST’S REVIEW • SEPTEMBER 2014 • REPRINT
Copyright © 2014 by A.M. Best Company, Inc. All Rights Reserved. Reprinted with Permission. www.ambest.com

3. Thereafter, the validator prepares
a model validation report that clearly
details the testing performed;the find-
ings from the validation; recommen-
dations for improvement;a prioritized
ranking of next steps; and limitations
of the validation.
An executive summary is often
prepared for senior management to
summarize key findings.
Models that contain issues can be
updated,replaced or restricted to cer-
tain purposes, depending on the
company’s risk limits and material-
ity thresholds.
Likewise, any issues related to gov-
ernance of the model could lead to
changes in model governance policies.
When models are changed, this
marks the beginning of a new mod-
eling cycle of development and
implementation (which includes
documentation, testing and approv-
als),use and validation.
Why MRM Matters
Models are now intrinsic to the
operations of insurance compa-
nies. Their misuse or errors can
have a significant, harmful impact
on an insurer. And over time, mod-
els have become increasingly com-
plex, making it even more impor-
tant that insurers pay attention to
model risk.
Leading insurers are developing
MRM frameworks that include mod-
el governance, development and
implementation, and validation to
mitigate model risk in response to
external forces such as regulators,
or proactively as companies become
more aware of the potential impact
of model risk.
Actual or perceived conflicts of
interest can diminish an insurer’s
efforts, so it is important for objectivi-
ty to guide MRM activities.When done
right, MRM can provide confidence
to internal and external stakeholders
that a company’s models are sound
and that their results are reliable. BR
Inside and Outside Influences
Model risk management typically is driven by some or all of these factors:
• Increased frequency of financial restatements due to model errors.
• Ineffectiveness of some models connected with the recent financial crisis.
• Sarbanes-Oxley requirements assigning management responsibility for financial statements.
• Own Risk and Solvency Assessment requirements developed by the National Association of
Insurance Commissioners for insurers to self-assess their risk management adequacy and
prospective solvency under normal and severe stress scenarios.
• Board of Governors of the Federal Reserve System SR 11-7 Supervisory Guidance on Model
Risk Management.
• Solvency II and its Internal Model Application Process.
• The North American CRO Council’s Model Validation Principles Applied to Risk and Capital Mod-
els in the Insurance Industry.
• Greater focus on models by internal and external auditors and regulators.
• Management’s desire to understand model limitations and weaknesses.
Source: Towers Watson
Copyright © 2014 by A.M. Best Company, Inc. All Rights Reserved. Reprinted with Permission. www.ambest.com
BEST’S REVIEW • SEPTEMBER 2014 • REPRINT 3