An Easier Way to Understand the Effectiveness of Risk Controls

KRIs, KPIs, ORSA, ISO, COSO…risk controls, risk owners, risk appetite.

The acronyms, the alphabet soup, oh my!

To anyone with little to no experience, risk management jargon can be dizzying and confusing, especially to executives who are often deluged with risk registers, reports, and processes that are overwhelming and not helpful for managing the organization for success. While unfortunate, it should not come as a surprise that fewer than 1 in 5 organizations believe their risk management processes provide a unique competitive or strategic advantage.

After all, executives are not interested in jargon and processes. What they’re most interested in knowing is whether they are on track to meet goals.

Key risk indicators (KRIs) focused on objectives help managers and executives understand this, but when used to their full potential, they can also help the company see what’s coming down the road and take steps to address future uncertainty. Taking proactive steps like this is one of several key differences between traditional and enterprise risk management.

A recent webinar from RIMS and Resolver explored another indicator organizations can use known as key control indicators, or KCIs. In layman’s terms, a KCI is a metric to understand the effectiveness of risk controls, which are actions taken to mitigate the negative impacts of risks.

Other metrics like key performance indicators (KPIs) and KRIs can be lagging and leading indicators. KCIs fall in the middle. In the webinar, Chase Clelland, VP of ERM at Grow Financial Credit Union, provides the following example of what indicators look like for the identified risk of loan defaults:

  • KPI = number of loans for clients who have past defaults
  • KCI = number of clients with insufficient collateral cover
  • KRI = number of loans who have past defaults and do not have sufficient collateral cover

Upper and lower thresholds can be established around each of these indicators that decision-makers can use to change goals, or in the case of Chase’s example, lending decisions or collateral requirements. A post from earlier this year on developing KRIs includes a couple of graphs showing how thresholds around revenue can influence decisions.

Like KRIs, KCIs shouldn’t be developed until after a risk’s likelihood, impact, and any other dimensions have been fully assessed and understood. And like KRIs…

Developing true KCIs to monitor the effectiveness of risk controls can be challenging and should only be done by organizations with robust risk management capabilities.

When asked about the biggest barriers to having effective metrics, Chase identifies the three biggest challenges he has faced: identifying the right metrics to monitor, knowing which metrics to evaluate over time, and having insufficient internal processes.

But in order to have a true KCI, you must consider both inherent and residual risk in the following way – Inherent risk – Control effectiveness (KCI) = Residual risk.

Merriam-Webster defines inherent as:

…involved in the constitution or essential character of something: belonging by nature or habit.

When it comes to risk, it can be complicated to understand what “inherent” means since it requires the manager or executive to imagine an alternate reality where no mitigations or other risk management activities take place.

Therefore, it can be extremely complicated and challenging to understand the organization’s inherent risks, making it difficult to have an accurate residual risk assessment.

So how can I understand the effectiveness of risk controls without complicating things?

Like ERM in general, KRIs and KCIs may sound simple in theory but are far from easy.

When asked about measuring the effectiveness of a control, co-presenter Terry Lampropoulous, Professor of Risk Management at Seneca College, says that it varies by company, industry, and even country since regulations can differ.

Besides testing by the audit group or some other independent party, paying close attention to losses or other indicators can indicate a control is not working. Chase explains that if you see losses going up, you either have “…controls that are not properly in place or a risk assessment that is way off.”

To add to Terry and Chase’s comments, true KCIs may not be necessary or even practical considering the challenges of understanding inherent risks.

The truth is you’re going to have different types of controls, whether they are systems-based, process-based, manual or automated. Besides testing and auditing by a third-party, having a clear understanding of the objectives and carefully monitoring the risk each control is linked to is the simplest way to understand the effectiveness of risk controls.

If it appears like a risk event is materializing or performance metrics are not being met, it may be a sign that the controls need to be changed or the person(s) responsible for implementing them is not fulfilling their obligations.

Although the webinar was quite informative, it was still very risk-centric, or focused on reducing negative consequences and preventing failure. As I explained earlier and in other posts, the point of KRIs and thresholds is not always to prevent a negative situation, but to indicate where additional risk can be taken in a responsible way.

It’s important to proceed with caution when developing metrics, especially a KCI. If your company is not equipped to process, analyze, and take action with the data, the whole process can be more trouble than it’s worth.

A more effective course of action is to understand what your goals are, what can prevent you from achieving those goals, carefully monitoring the steps your company is taking, and adapting your approach as conditions warrant.

How do you measure the effectiveness of risk controls in your organization?

I’m interested in hearing your thoughts on this topic, and I know others in our profession are too. To share your experience, questions, or concerns, please leave a comment below or join the conversation on LinkedIn.

And if your company is struggling to understand how to better assess risks or adapt to ever changing circumstances, please don’t delay – reach out to discuss your situation today.

, , , ,

Related Posts

2 Comments. Leave new

  • I believe your formula is incorrect.

    Inherent risk – Control effectiveness (KCI) = Residual risk.

    This is correct:
    Inherent risk – Amount of risk mitigated by a control/s = Residual risk.

    KCI is….
    An indicator used by organizations to define its controls environment and monitor levels of control relative to desired tolerances. KCIs answer the question: ”Are our organisation’s internal controls effective? Are we ‘in control’?”

    KCI only tells you about the control, not about the risk is it mitigating.

    • Hi Thomas:

      Thank you for reaching out. I partially agree – a KCI is meant to tell us about the effectiveness of the control, but we can also glean insights about the risk itself by how well the KCI is working to improve (make better) the risk’s impact and likelihood.

      As to the formula, it was provided as part of the RIMS webinar, and as far as I could tell, it was only meant to provide a generic way to understand inherent and residual risk. As I explain in the post though, it can be difficult to define what the inherent risk is since it requires you to picture what the risk would look like if no mitigations or controls existed. This is why I added my observation that true KCIs may not be necessary or even practical. I think your formula suggestion, though, is much more in-depth than the one provided in the webinar. Thank you again for your comment and additional suggestions!


Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.