Expert Commentary

Measuring and Managing Operational Risks

Risk managers have become in desperate need of reliable methods for measuring and managing operational risks. In this first in a series of articles, Samir Shah describes several promising methods for quantifying operational risks.

Quantitative Methods
April 2002

The risk management industry has seen a tremendous surge in interest in measuring and managing operational risks. This outpouring is a result of a combination of recent regulatory developments in corporate governance and capital adequacy, and a growing realization that an enterprisewide view of risk management is simply good business. The wave of recent well-publicized corporate failures has shown that, more often than not, the culprit was an operational risk—for which no capital is held—rather than market, credit, or insurance risks.

In response, regulators in Canada, the United Kingdom, and Australia have revised corporate governance standards to hold directors responsible for managing all risks: market, credit, insurance, legal, technology, strategic, regulatory, etc. The Basel Committee has proposed an operational risk capital charge for banks to protect against "…failed internal processes, people and systems or from external events." Risk managers have become in desperate need of reliable methods for measuring and managing operational risks.

This series of articles will describe several methods that are promising candidates for quantifying operational risks.

Characteristics of Operational Risks

Before we can talk about modeling operational risks, it's useful to first understand the unique characteristics of operational, or "op" risks and their implications on modeling methods.

Characteristic of Op Risks Implication
1. Op risks are endogenous, i.e., specific to the facts and circumstances of each company. They are shaped by the technology, processes, organization, personnel, and culture of the company. By contrast, market, credit and insurance risks are driven generally by exogenous factors. Need to gather company-specific data. However, most companies don't have a long history of relevant data. In banking, industrywide data is being gathered, but it may not be representative.
2. Op risks are dynamic, continuously changing with business strategy, processes, technology, competition, etc. Even a company's own historical data may not be representative of current and future risks.
3. The most cost-effective strategies for mitigating op risks involve changes to business processes, technology, organization, and personnel. Need a modeling approach that can measure the impact of operational decisions. For example, "how will op risks change if the company starts selling and servicing products over the Internet, or if a key function is outsourced?"

The endogenous and dynamic nature of op risks suggests a greater reliance on expert input and professional judgement to fill data gaps—at least until companies gather enough historical data over varying business environments. Use of operational strategies to mitigate op risks suggests a causal modeling approach that managers can use to perform "what-if" analyses. After all, the goal of risk management is to reduce op risks, not just measure them.

Risk Modeling Methods

There is a continuum of methods to model risks (see Figure 1). Although there are many ways to classify these modeling methods, for our purpose it is useful to organize methods based on the extent to which they rely on historical data versus expert input. This list of methods is by no means exhaustive. However, it illustrates very nicely that there is large inventory of risk modeling methods across finance, engineering, and decision science disciplines that can be drawn on to suit a particular circumstance.

Figure 1

Risk Modeling Methods Graph
Figure 1. There is a continuum of risk modeling methods that vary in their relative reliance on historical data versus expert input. Each method has advantages/disadvantages over the others and requires varying skills. A method should be chosen to match the specific facts and circumstances.

Methods Based on Statistical Analysis of Historical Data

Market, credit, and insurance risks rely heavily on statistical analysis of historical data for quantification. These risks are modeled primarily by using methods on the left side of Figure 1. These include, for example:

  • Actuarial approaches based on convoluting frequency and severity probability distributions
  • Simulation using stochastic differential equations
  • Extreme value theory to model the tail of a probability distribution

Operational risks can also be modeled using these methods, when there is adequate amount of representative historical data. High-frequency, low-severity op risks, such as bank settlement errors for example, usually generate enough data to use methods based on statistical analysis. Although even in this example, as banks implement straight-through-processing (STP), the risk will change, and the historical data may not be a reliable indicator of prospective risks.

Methods Based on Expert Input

Decision scientists have long relied on methods listed on the right side of Fig. 1 to quantify risks when there is little or no objective data. They have had to rely almost exclusively on expert input to quantify risks, such as likelihood of success or failure of a new drug in early stages of research. These include:

  • Delphi method to elicit information from a group of experts
  • Decision trees, which lay out decision points and resulting discrete uncertain outcomes
  • Influence diagrams, which also map out cause-effect relationships

Over time, they have refined these methods to minimize the pitfalls and biases arising from estimating subjective probabilities, thereby increasing the reliability of these approaches.

Methods Based on a Combination of Data and Expert Input

The methods listed in the middle of Figure 1 rely on a combination of historical data, to the extent it's available, and expert input as needed to fill data gaps. They include, for example:

  • Fuzzy logic, which uses linguistic variables and rules based on expert input
  • System dynamics simulation, which uses non-linear system maps to represent the causal dynamics of a system
  • Bayesian Belief Networks (BBN), which rely on a network of cause-effect relationships quantified using conditional probabilities

Most of these methods are borrowed from other disciplines, primarily the engineering sciences.

As in the case of Goldilocks, for op risks, "The statistical methods require toooo much data," "The decision science methods rely toooo much on expert input," and "The methods in the middle are juuust right!" These methods offer the best match to the unique characteristics of op risks.

As businesses have become more complex and the interdependencies have increased, managers have struggled to maintain control and make decisions under uncertainty. Use of enterprise data warehousing and data mining has substantially increased the amount of data that is available to managers. However, the sad truth is that the terabytes of data have not significantly increased their understanding of the enterprisewide business dynamics.

The complexity of the systems is increasing at a faster rate than our knowledge of it. Managers have responded by focusing on smaller areas of their business and becoming more specialized. They have a much deeper understanding of their domain but a much lesser understanding of how their domain interacts with others.

Modeling techniques need to be flexible enough to consolidate knowledge that is fragmented across many experts. They also need to effectively leverage both data and expert input in order to develop a clearer and more reliable representation of reality.

Description of Specific Risk Modeling Methods

The following methods for measuring and managing operational risks are described in detail in separate articles. Please click on a method to view other articles.

Opinions expressed in Expert Commentary articles are those of the author and are not necessarily held by the author's employer or IRMI. Expert Commentary articles and other IRMI Online content do not purport to provide legal, accounting, or other professional advice or opinion. If such advice is needed, consult with your attorney, accountant, or other qualified adviser.

Like This Article?

IRMI Update

Dive into thought-provoking industry commentary every other week, including links to free articles from industry experts. Discover practical risk management tips, insight on important case law and be the first to receive important news regarding IRMI products and events.

Learn More


Social Media

User ID: Subscriber Status:Free