of this report is to give an overview of Model Risk Management (MRM) in the
increasing dependence on models pushes model risk to be one of the top concerns
that insurers hold. The model risk lies in model misuse and flawed models. The
constituent elements of MRM system are the model classification, model
development, and model validation. Model classification assigns model a risk
rating in accordance with risk exposure and a qualitative evaluation of model
design and limitations. Acting as the first line of defense, the model owner is
responsible to take corrective actions towards model deficiencies and to ensure
the model is performing as intended. Model validation reassures that model
aligns with the business objectives and regulatory standard.
MRM system is
expected to experience three evolutionary stages. The risk function first builds
a framework encompassing key elements and then transits to implementing a
robust MRM program. Ideally, the MRM program creates value by incorporating
with corporate strategies. The
inefficiency of MRM emerges from inadequate MRM performance evaluation,
regulatory guidelines, and poor model submission.
The number of models is rising drastically at both life
and Property/Casualty insurance companies, as they have utilized models for an
ever-broadening scope, ranging from pricing, underwriting, and asset-liability
matching to strategic planning and customer relationship management.
Over the years, insurers have made great efforts to build a sound MRM
framework, which is also central to enterprise risk management activities. With
advanced analytics and digitization, more complex models are developed to
inform business decision-making. A negligible model error is very likely to
lead to a substantial financial loss and even reputation damage.
is intended to introduce the existing MRM practices and touches on some
outstanding issues in the system. A three-stage evolution of MRM is
demonstrated to summarize the journey from initiating MRM framework to value extraction.
2.0 Model Risk
framework boils down to what is a model risk. In the SR 11-7: Guidance on Model
Risk Management, the Federal Reserve and Office of the Comptroller of Currency
OCC (2011) defines model risk as follows: “The use of models invariably
presents model risk, which is the potential for adverse consequences from
decisions based on incorrect or misused model outputs and reports”. It can be obviously
seen from the definition that model risk is inherited and unavoidable.
As further revealed
in SR 11-7, model risk primarily arises from defective models and model misuse
(Federal Reserve & OCC, 2011). Defective models include model specification
errors, which are fundamental flaws in model design, such as mistakes in
methodology, and omission of key assumptions. A model may also be used in ways deviating
from the original intent or there could be misconceptions about methodology
limitations and assumptions.
risk grows with higher complexity, wider scope and increasing uses. The devastating impact of model risk is illustrated
by the collapse of Long-Term Capital Management (LTCM), a once prestigious
hedge fund management firm. LTCM’s main strategy is to make convergence trading
by finding mispriced securities, then taking long positions in the cheaper ones
and short positions in the valued ones based on the sophisticated computer
models (Sungard, 2011). Unfortunately, the models, not subjective to strict
model validation and stress testing, mistakenly projected the correlation
between the long and short positions and escalated LTCM’s debacle.
2.1 Model Classification and Development
like MRM was only at the outset a few years ago. But now, financial institutions, in
particular, insurance carriers and banks, have taken bold strides in advancing
underlying element in MRM framework, model classification assigns a model risk
score or rating to a model. The risk score is derived from the severity of
model output, also known as materiality, and a qualitative assessment of the model
with respect to but not limited to model use, model scope, assumptions, key person
risk, quality of documentation, and software pitfalls.
development suggests that risk mitigation should be throughout the model life cycle
from initial design to implementation and monitoring. Model owners and users
ascertain that model development aligns with the intended model usage. As an
additional layer for model development, User Acceptance Testing is performed to
evaluate the overall function of the model, scrutinize model inputs, and keep
the model abreast of errors.
It is worthy
to note that documentation is often overlooked during model development. In a
published report, “Making model risk management more cost effective”,
PwC (2017) stressed that documentation is of critical importance and is the
starting point of model vetting. The documentation that the model owner
prepared need to be complete and detailed. The level of detail should be sufficient
for a validation team to check conceptual soundness and replicate the
functionality of the model.
2.2 Model Validation
According to the Federal Reserve and OCC (2011),
“Model validation is
the set of processes and activities intended to verify that models are
performing as expected, in line with their design objectives and business
uses”. Model risk management is a good position when the
validation experts do not have a stake in model development or model use and
are not involved in the decisions made upon the model.
The level of risk determines the level of validation,
with higher-risk models prioritized for full validation and lower-risk models
assigned with a light review. Full
validation calls for an independent professional team to review the model
specification, assumptions, input, processing, output, reports, and controls
with great attention. Model replication could be built on other platforms such
as Excel and Python to ensure the model is performing as intended.
An effective validation identifies model limitations
and continues on an ongoing basis to keep track of existing risks and be wary
of emerging ones. To be agilely responsive to the changing regulation,
financial and economic environment, periodic reviews should be conducted to
determine if the model deviates from the original objective and if the
limitations are still applicable.
3.0 The path to
Model Risk Management
written by Crespo, Kumar, Noteboom, and Taymans (2017) presents three
evolutionary stages of MRM. The first stage is to establish a basic framework
for MRM. This involves setting policies and guidelines for MRM objectives,
governance, model inventory, model validation, reporting, and compliance. As a
starting point, two functions dominate MRM infrastructure. On one hand, a
governance team builds and maintains MRM requirements together with standards
that are consistent with industry practice. On the other hand, a validation
team applies rigorous vetting to assure the conceptual soundness of models, identify
and gauge the risk level against regulation. Overall, the MRM function at this
stage is mainly backward-looking and operates within model risk review
phase, where most North American insurers are standing, is to execute a robust
MRM program. At this point, insurers usually have the adequate capacity to
develop their own risk controls and internal metrics, aligning with the
industry standard. Model Risk Management will further stretch to embed the core
value of risk management into stakeholders across the enterprise. It focuses on
transforming model risks to strategic input for senior management.
mature MRM function not only brings insights and foresights to corporate decisions,
but earns efficiencies by proactively maintaining a high-performing model
inventory. Though the future is unforeseeable, the strong ability to detect
early-warning signals and draft plans enables insurers to respond quickly in
stressed conditions. At the ultimate stage, MRM serves as a strategic thought
partner and extracts value from better decision making.
3.1 Challenges in Model Risk Management
It is every
insurer’s ultimate goal to arrive at the third stage. But in reality, MRM
serves more like lines of defense towards model risks and aims to meet
regulatory and corporate compliance. Unlike pricing, where the success can be
measured by annual sales or new business values, the effectiveness of MRM is
hard to define and less palpable in the short run.
PwC (2017) supports that model risk is more akin to business and strategic
risk, which cannot be eliminated by assigning capital requirement to it. Compared
with a proliferation of regulatory framework for capital adequacy, liquidity
risk and statutory reserve, such BASEL III, Solvency II, International
Financial Reporting Standards (IFRS), and Generally Accepted Accounting
Principles (GAAP), there is lacking industry standards for MRM. One of the most
prominent standards is SR 11-7, issued in 2011, only provides very high-level
supervisory support and general principles for insurers and banks.
poor model submission has become an outstanding concern for MRM team. Ill-prepared
documentation, delayed model delivery and increasing waiting time for model
owners’ response gigantically reduce efficiency and pose threats to model
inventory. To alleviate the dilemma, a
highly integrated risk culture has to be cultivated across all business units.
An early definition of model risk in SR 11-7 implies that model risk,
stemming from the inappropriate use of models, is inevitable. In addition to
misuse, model risk possibly lies in model misspecification, such as parameter
errors and inapplicable assumptions. An important lesson learned from the
failure of LTCM is that even the most complicated model is vulnerable to model
risks and has a destroying power to business.
Under the Model Risk Management framework, model classification combines
quantitative and qualitative criteria to evaluate the model risk. Risk
mitigation plans should be implemented throughout the model life cycle. Model
owners are responsible to confirm the correctness of the model and provide
well-prepared documentation for an independent validation. Effective model
validation should allocate resources to models based on the risk level and
importance to strategic decisions.
The path to the maturity of MRM is transformative, which can be
described as three phases: establishing a framework, exerting a robust MRM
function, and capturing value from the system. MRM presents several challenges
to insurers. The measurement of a successful MRM is still underway. The nature
of model risk, developing industry guidance and unsatisfactory model submission
will cut off the benefits MRM brought.
Even though we know where an MRM program optimally should go so as to
make the best out of business operations, it will take enduring efforts to get