Modeling the Reality of Risk: The Cornerstone of Enterprise Risk Management
July 2001
A major reason managers are frustrated with
their progress on ERM is because they don’t have adequate risk modeling tools.
Learn why standard statistical models don't work well for operational risks,
but structural models do, because they capture the causal relationship among
the variables.
by Jerry
Miccolis and Samir Shah
Tillinghast-Towers Perrin
In our earlier articles in this series on enterprise risk management (ERM),
we pointed out that a major reason managers are frustrated with their progress
on ERM is because they believe they don't have adequate tools. This is particularly
the case when it comes to risk modeling, developing probability distributions
of outcomes that represent the uncertainty associated with specific risk factors.
The Problem with Standard Statistical Modeling Tools
The risk modeling tools most used today are based on statistical methods.
Managers generally know that the tools are "imperfect" for modeling operational
risks. But in our view, the standard statistical modeling tools are also inadequate
for modeling financial risks. In fact, they expose managers to the very risks
they were trying to avoid by using the models—poor decision-making. Fortunately,
there are better tools available: "structural" models of risk that capture cause-and-effect
relationships between risk factors and outcomes. Using structural risk models
enable managers to develop, with confidence, appropriate risk mitigation strategies
for their organizations.
Statistical tools look like plausible
approaches to modeling enterprise-wide risks because they use hard data captured
in databases over fairly extensive periods of time. Financial managers have
used these tools to model individual financial risks beyond the direct control
of their organizations. These include such macroeconomic risks as changes in
interest rates and exchange rates, and asset performance. They also include
insurable risks, such as mortality and property/casualty claims.
When financial managers, especially in banking, have been challenged to treat
financial risks more holistically, their approach has generally been to simply
aggregate the risks. That is, they build models of each financial risk separately
and then combine the risks using a statistical approach.
For example, if the risks were interest rates, equity returns, and liability
volatility, these financial managers would build statistical distributions representing
each of the risks and then mathematically combine (i.e., convolute) the distributions.
To do that means specifying the form and parameters of each of the three risk
distributions and the nature of the linkage between them. If the probability
distribution for each risk is part of a family of distributions with special
mathematical characteristics—such as symmetric distributions with constant covariance—the
manager determines the aggregate distribution by simply doing the math.
That couldn't be simpler, faster, or easier to implement. But it also can
lead to incorrect decisions, as we will discuss in a moment.
Using Statistical Modeling Tools with Operational Risks
What is not so simple, managers find, is using the statistical modeling tools
with operational risks. Those are the risks that arise from such things as the
entry of a new product or company into a market, poor business judgment on the
part of a senior manager, or the decision to use a new product distribution
system such as the Internet (or even direct telemarketing). Financial managers
who are comfortable with using statistical tools to model financial risks find
themselves frustrated when trying to use those tools with these sorts of risks.
The problem, they say, is that there is not enough historical data on operational
risks to build valid statistical models. The solution, they say, is to start
building databases of operational risks—and many of them, especially in the
banking industry, have begun to do exactly that.
In our view, they will never get there on that horse alone. That's because
the problem is not with the richness of the data, but with the adequacy of the
tool. Statistical tools don't work for operational risks because those risks, by their nature, do not lend themselves to a simple
statistical description. Moreover, the inadequacy of statistical tools for modeling
operational risks actually reveals the inadequacy of those tools for modeling
integrated financial risks. In both cases,
the statistical tool fundamentally doesn't work because it oversimplifies reality.
As Einstein once said, "Solutions should be simple, but not too simple."
Take the integration of financial risks, for instance, which many financial
managers assume can be represented by aggregating the individual financial risks
as we described above. If you think about it a moment, it is pretty clear that
reducing the interrelationship between two risks to a single number—their correlation—is
too restrictive to capture the nature of most risks in the real world.
For example, consider the behavior of equity markets around the world. At
most times, the behaviors of the stock markets in New York, London, and Tokyo
are only partially correlated. While they react to one another, most of the
time they also react to local conditions. But sometimes they do move in almost
lockstep, when, for example, one of the markets declines precipitously. The
usual statistical models do not and cannot capture this sort of behavior; they
are simply incapable of representing the varying levels of correlation among
the markets that depend on different, complex circumstances.
The Limitations of Using Statistical Models with Operational Risks
The weakness of statistical models is even clearer in the case of operational
risks, again because of their fundamental nature. In the first place, operational
risks vary significantly, based on how a company manages its internal operations,
so the data needed to apply standard statistical approaches would need to be
company-specific. It couldn't merely be industry-specific. Moreover, the data
should be representative of the current operations environment. Because these operations are dynamic, changing with
adjustments to the basic business model, as well as technology and work processes,
the likelihood of gathering sufficient representative data is fairly remote.
Second, operational risks are managed through changes in processes, technology,
people, organization, and culture. They are not generally managed by using financial
tools such as hedging in the capital markets. Managers want to know how operational
risks would change if they were to implement alternative operational decisions.
It's highly unlikely that historical data—the foundation of statistical approaches—will
be, or can be, segmented on that basis.
Third, operational risks are of two sorts: event risks and business risks. Event risks are
isolated occurrences that generate loses, such as a technology failure. Business
risks are created by business decisions. Although it might be possible to gather
sufficient historical data on event risks to build a rough statistical model,
it is highly unlikely managers could do the same thing for losses that arise
from business decisions.
In short, the very nature of operational risks makes them ill suited for
statistical modeling. It's the problem with statistical modeling we saw in integrated
financial risk modeling writ large. For both financial risk modeling and operational
risk modeling, financial managers need much more flexible, robust tools. Those
tools are a family of approaches called structural modeling.
Structural Modeling Methods for Operational Risks
Structural methods differ from statistical models because they simulate the
dynamics of a specific system by developing cause-effect relationships between
all the variables of that system. Structural models can range from the very
mathematically rigorous, such as stochastic differential equations (particularly
useful in modeling complex financial risks), to methods that rely on a mixture
of mathematical calculations and expert opinion, such as system dynamics simulation,
fuzzy logic, and Bayesian belief networks (BBNs). These latter methods are especially
useful for modeling operational risks.
Using structural methods for modeling financial risks solves the problem
of oversimplifying complex, variable relationships. For example, financial managers
can model the complexity of a wide range of macroeconomic risks—including short-
and long-term interest rates, real GDP, price and wage inflation, equity earnings
yield and the like—by representing their interaction in a "cascade" structure
in which each variable in the cascade is dependent on the variables "above"
it.
That arrangement captures the causal relationship among the variables. And,
luckily, experts have already figured out most of the macroeconomic relationships
higher up in the cascade. The task of the financial manager is to link these
relationships to those company-specific risks at the bottom of that organization's
cascade.
Structural approaches to modeling financial risks are especially useful in
a multi-period context. A stochastic scenario generator, for instance, can simulate
internally consistent paths for interest rates, inflation, equity markets, currencies,
and the like over multiple time horizons. Using structured equations, managers
can induce appropriate levels of mean reversion, spread reversion, etc. The
approach is clearly superior to assuming that these variables behave in a "random
walk" over time.
Generating scenarios using a structural model eliminates the constraints
on risk modeling associated with statistical models, such as limitations on
the form of probability distributions and constant correlation. Thus, structural
modeling provides a much more reliable representation of the nature of financial
risks and their interaction. It allows financial managers to make much sounder
decisions, decisions based on how financial variables actually behave in the
real world.
System Dynamics Simulation
As useful as structural methods are for capturing the reality of financial
risk, they may be the only way to model the reality of operational risks. System
dynamics simulation, developed in the 1950s by Dr. Jay Forrester at MIT, is
a particularly robust structural method and illustrates the value of all such
approaches.
The starting place for a system dynamics simulation is expert "testimony,"
which overcomes the problem of the lack of historical data. The model builders
gather information from experts in a given domain in order to develop a graphical
system map of the cause-effect relationships among the key variables to represent
the dynamics of a specific risk. A system map for the risk associated with the
decision to use the Internet as a distribution channel to launch a new product,
for instance, would capture the impact of variables such as brand name, marketing,
and advertising expenditures, complexity of product features, use of a financial
services portal, process cycle times, availability of on-line support, and the
like.
Then, the model builders quantify each cause-effect relationship, using a
combination of available historical data and expert input, to again overcome
the problem of gaps in the historical data and to adjust for data that may not
be representative. To the extent that the expert input is uncertain, the cause-effect
relationship is represented as a probability distribution around a point estimate.
Next, the model builders run simulations to develop a range of outcomes for
key operational and financial variables. The output across the simulation runs
is summarized as a probability distribution of financial variables. The probability
distributions represent the operational risk, given the current operating environment.
Finally, the model builders perform "what-if" analyses by modifying the decision
variables that represent changes in operations and then rerunning the simulations.
This step allows the managers who are directing the model building to evaluate
the effect of alternative decisions on operational risk. They can then make
and implement the decision that will be most likely to get them the outcome
they want, at the risk level they and their organization can tolerate.
Other Advantages of Using Structural Models with Operational Risks
Structural models have other advantages, over and above avoiding the limitations
of statistical methods. These include facilitating—or "forcing"—interaction
among managers in an organization who normally don't think about how their individual
decisions affect one other and the organization as a whole, as well as focusing
the members of an organization on the specific kind of data they actually need
to clarify the assumptions on which they base their key decisions (a much more
cost-effective approach than trying to "learn everything about everything").
But, from the broad standpoint of enterprise risk management, the fundamental
value of structural modeling is that it removes the biggest barrier to managing
risks holistically: it's the tool managers have been looking for.
For more on this subject, please refer to the following monographs, available
on a complimentary basis from Tillinghast-Towers Perrin: Enterprise Risk Management:
An Analytic Approach [Publications]; RiskValueInsights(tm):
Creating Value Through Enterprise Risk Management—A Practical Approach for the
Insurance Industry. For more information, go to the Tillinghast Web site.
Opinions expressed in Expert Commentary articles are those of the author and are
not necessarily held by the author’s employer or IRMI. This article does not purport
to provide legal, accounting, or other professional advice or opinion. If such advice
is needed, consult with your attorney, accountant, or other qualified adviser.