Skip to main content
Blog Post

Risk managers: Beware of the right side of the decimal

Riskhanteringsverktyg & -teknologi|Doradztwo ubezpieczeniowe i technologie|Reasekuracja
N/A

By Alice Underwood | September 5, 2019

The goal of risk management should be to achieve a better business result – not the best analyzed business result.

Recently, a client executive expressed to me the concern that the company’s economic capital modeling team was “spending too much time on the right-hand side of the decimal point.” Unfortunately, I knew all too well the kind of thing that was likely going on.

Unlock More

About our ‘A Year in the Life of the Strategic CRO’ series

In our ongoing A Year in the Life of the Strategic CRO series, risk experts from our Insurance Consulting and Technology team, Willis Re and other parts of Willis Towers Watson cover how how a strategically focused CRO can drive corporate strategy through the enterprise risk management planning process and throughout the year.

In the business world, there are very few analyses that can meaningfully distinguish between 9.5370559% and 9.54%. And there are very few risk management questions that depend on more than three – or perhaps two – significant digits. I don’t think I have yet dealt with an insurance company whose leadership would make a different decision when presented with an opportunity yielding 9.54% return on risk-allocated capital (RORAC) as compared to 9.5% RORAC.

Real precision or false precision?

One type of false precision is failing to round to significant digits, of course – but another type of false precision is expending a lot of time and energy on making tweaks to an analysis that are very unlikely to ever affect the significant digits of the result.

This tendency might be especially acute for those of us who have come through a quantitative educational and career path. Mainstream educational systems reward students for delivering the “right” answers, and lavish praise on straight-A students. Credentialing exams, such as those required for actuarial, accounting and financial advisory designations foster reverence for a Platonic ideal of correctness.

We’ve been trained to be perfectionists.

Yet the world in which we live, and the business environments in which we operate, are messy and imperfect. That can be a very uncomfortable reality. It can be tempting to focus on aspects of the work we feel we can control – and to carve out little pristine bubbles where we can polish and perfect things to our heart’s content.

Driving business results or entertainment?

My colleague, Dave Ingram, wrote about one potential outcome of this tendency: he called it a risk management entertainment system. As Dave pointed out, it’s possible to keep oneself very busy constructing more and more elaborate models that take the analysis further and further to the right of the decimal point.

While that may be entertaining for those of us with a quantitative bent, it’s unlikely to drive business results. We owe it to ourselves – and to both our principals and our principles – to consider the level of precision or number of significant digits that will be meaningful. Even risk professionals who are not actuaries may find the Actuarial Standards of Practice helpful in this regard.

Actuarial Standard of Practice No. 41 from the Actuarial Standards Board addresses “Actuarial Communications.” Section 3.4.1 of this standard says: “the actuary should consider what cautions regarding possible uncertainty or risk in any results should be included in the actuarial report.”

Actuaries and other risk professionals know that any point estimate of a future quantity is almost certain to be inaccurate. That’s why the term “reasonable range” appears so often in the actuarial literature. An estimate that cannot be made with a reasonable degree of accuracy is not helpful. But conversely, when the uncertainty inherent in a particular method of estimation can be reasonably quantified, that’s useful information. Sensitivity testing can be one practical way of assessing uncertainty.

Focusing on RORAC

Suppose that I am analyzing the RORAC of a proposed deal d. Any number of parameters may well affect the calculated RORAC, but for purposes of this discussion let’s focus just on the risk-free interest rate r, and write:

RORAC(d, r) = Return on risk-allocated capital of deal d at risk-free interest rate r

Perhaps my best estimate today is r = 2%, but I wouldn’t be terribly surprised if the risk-free rate applicable over the life of the deal turned out to be 2.2% or 1.8% instead. Furthermore, suppose that my raw calculation yields:
RORAC (d, 2.2%) = 9.4925738%
RORAC (d, 2.0%) = 9.5370559%
RORAC (d, 1.8%) = 9.5818582%

In this case, if asked for a point estimate, it’s not clear there would be any benefit in giving an answer any more precise than “the estimated RORAC of the deal is roughly 9.5%.”

How should one approach sensitivity testing? That’s a much bigger question than a single blog post could hope to answer, but in general I suggest focusing on a handful of the most important parameters (such as the risk-free interest rate) that affect the result and then test the effect of varying them across a reasonable range.

What’s a ‘reasonable’ range?

What’s “reasonable”? Well, that depends on the question being asked and the context in which a given answer will be used. The analyst needs to apply common sense and good business judgment – and also consider the intended use.

For example, if my principal is targeting a RORAC in the “high single digits” and the deal I’m analyzing above is one of many in a portfolio, it hardly seems sensible to refine my analysis any further.

On the other hand, if my principal has promised her stakeholders that she will exceed 10% RORAC over the coming year, and the deal I’m analyzing could dominate the results of her business, I might want to spend more time looking at sources of uncertainty other than the risk-free interest rate, and once I am comfortable with the analysis sensitivities I might say something like “my estimates range from 9.49% to 9.58%, which could imply an overall portfolio RORAC of X% to Y%.”

But it’s very difficult to imagine a circumstance in which it would be sensible for me to either:

  • Spend time refining my estimate of a secondary parameter which, when varied over a reasonable range, yields RORAC values from 9.5368% to 9.5373%
  • Tell my principal that the RORAC of this deal equals 9.5370559%

Sometimes good enough is actually better

No company can afford to have highly-paid professionals spend most of their time building more complex ways to analyze non-significant digits.

Instead, a much better use of our time as ERM professionals is to ensure we understand what business actions and decisions depend on our work. With those ends in mind, what level of precision is needed? The strategic CRO will consider the degree of precision embedded in the company’s strategic planning process – and prioritize their efforts accordingly.

Understanding – and limiting our work – to a level good enough to enable a sound decision isn’t slipshod. Good enough is, by definition, good enough. Even when it’s not perfect.

As my client pointed out, the goal of enterprise risk management isn’t to achieve the best analyzed business result – it’s to achieve a better business result.

Previously in the A Year in the Life of the Strategic CRO series: How inclusion and diversity plays a critical role in risk management.

Author

Alice Underwood
Global Leader of Insurance Consulting and Technology

Contact Us

Related content