Measuring and Managing Operational Risks

Characteristics of Operational Risks

Before we can talk about modeling operational risks, it's useful to first understand the unique characteristics of operational, or "op" risks and their implications on modeling methods.

The endogenous and dynamic nature of op risks suggests a greater reliance on expert input and professional judgment to fill data gaps—at least until companies gather enough historical data over varying business environments. Use of operational strategies to mitigate op risks suggests a causal modeling approach that managers can use to perform "what-if" analyses. After all, the goal of risk management is to reduce op risks, not just measure them.

Risk Modeling Methods

There is a continuum of methods to model risks (see Figure 1). Although there are many ways to classify these modeling methods, for our purpose it is useful to organize methods based on the extent to which they rely on historical data versus expert input. This list of methods is by no means exhaustive. However, it illustrates very nicely that there is large inventory of risk modeling methods across finance, engineering, and decision science disciplines that can be drawn on to suit a particular circumstance.

Figure 1

Figure 1. There is a continuum of risk modeling methods that vary in their relative reliance on historical data versus expert input. Each method has advantages/disadvantages over the others and requires varying skills. A method should be chosen to match the specific facts and circumstances.

Methods Based on Statistical Analysis of Historical Data

Market, credit, and insurance risks rely heavily on statistical analysis of historical data for quantification. These risks are modeled primarily by using methods on the left side of Figure 1. These include, for example:

• Actuarial approaches based on convoluting frequency and severity probability distributions
• Simulation using stochastic differential equations
• Extreme value theory to model the tail of a probability distribution

Operational risks can also be modeled using these methods, when there is adequate amount of representative historical data. High-frequency, low-severity op risks, such as bank settlement errors for example, usually generate enough data to use methods based on statistical analysis. Although even in this example, as banks implement straight-through-processing (STP), the risk will change, and the historical data may not be a reliable indicator of prospective risks.

Methods Based on Expert Input

Decision scientists have long relied on methods listed on the right side of Fig. 1 to quantify risks when there is little or no objective data. They have had to rely almost exclusively on expert input to quantify risks, such as likelihood of success or failure of a new drug in early stages of research. These include:

• Delphi method to elicit information from a group of experts
• Decision trees, which lay out decision points and resulting discrete uncertain outcomes
• Influence diagrams, which also map out cause-effect relationships

Over time, they have refined these methods to minimize the pitfalls and biases arising from estimating subjective probabilities, thereby increasing the reliability of these approaches.

Methods Based on a Combination of Data and Expert Input

The methods listed in the middle of Figure 1 rely on a combination of historical data, to the extent it's available, and expert input as needed to fill data gaps. They include, for example:

• Fuzzy logic, which uses linguistic variables and rules based on expert input
• System dynamics simulation, which uses non-linear system maps to represent the causal dynamics of a system
• Bayesian Belief Networks (BBN), which rely on a network of cause-effect relationships quantified using conditional probabilities

Most of these methods are borrowed from other disciplines, primarily the engineering sciences.

As in the case of Goldilocks, for op risks, "The statistical methods require toooo much data," "The decision science methods rely toooo much on expert input," and "The methods in the middle are juuust right!" These methods offer the best match to the unique characteristics of op risks.

As businesses have become more complex and the interdependencies have increased, managers have struggled to maintain control and make decisions under uncertainty. Use of enterprise data warehousing and data mining has substantially increased the amount of data that is available to managers. However, the sad truth is that the terabytes of data have not significantly increased their understanding of the enterprisewide business dynamics.

The complexity of the systems is increasing at a faster rate than our knowledge of it. Managers have responded by focusing on smaller areas of their business and becoming more specialized. They have a much deeper understanding of their domain but a much lesser understanding of how their domain interacts with others.

Modeling techniques need to be flexible enough to consolidate knowledge that is fragmented across many experts. They also need to effectively leverage both data and expert input in order to develop a clearer and more reliable representation of reality.

Description of Specific Risk Modeling Methods

The following methods for measuring and managing operational risks are described in detail in separate articles. Please click on a method to view other articles.

• Fuzzy Logic
• System Dynamics
• Bayesian Belief Networks (BBN)
• Actuarial Approach
• Stochastic Differential Equations (SDEs)
  • Scenario generation
  • GARCH modeling