KRIs and Real Risk Management — Objects in the Mirror May Appear Closer Than They Are
Eric Bonnell, SVP — Enterprise Risk Management
Key Risk Indicators (KRI) can be elusive, insufficient, and deceptive. Some are built from mathematics, some from experience. Many are artificial, unclear, and/or inaccurate, cobbled together to fit a predefined pattern that removes their effectiveness in relaying their message. All are subjective, misread, and prone to starting disagreements.
KRIs are only as good as their context, definition, perception, data structures, calculation, measurement scale, and visualization.
And what about the plethora of risks or threats that aren’t “Key”? How are they identified, managed, and communicated? Also, how do you account for the true risk picture while only referencing a one-dimensional KRI that is inclusive of only one key control, and myopic to the layers of defense in place? How much of the story is slipping through your fingers or otherwise ignored?
What is a KRI Anyway? And What Good is it?
A “Key Risk Indicator” or KRI is an approximating tool to help identify threats that are becoming or are likely to become real. Perhaps they should be called KTIs for this reason.
I believe that KRIs can be used to level set understanding and as a critical conversation starter. I also have seen the confusion and missed opportunities that result from putting too much emphasis on KRIs without additional insight (emerging threat analysis, “non-key” areas needing attention to manage risk, and recommendations to revise the KRIs being measured based on current needs).
KRIs can be qualitative or quantitative. Some leverage past outcomes as a way to measure risk (e.g., # delinquent loans, efficiency ratio, # of customer complaints related to fraud, etc.), others look to show control effectiveness (e.g., # of corrected access entitlements), and still, others try to show risk based on the level of information processes (# open issues related to technology).
KRI Examples and Concerns
Let’s look at the examples given above more closely and point out some concerns (spoiler: most of the concerns have to do with context and the inclusion of supporting information to start an effective management conversation with the correct subject matter experts):
- # Delinquent Loans — All things being equal, this is a good indicator of credit risk and operational effectiveness. But are all things equal? When the economy changes and more customers are prone to default, what will be the course of action to take? How do you know whether reducing the # delinquent loans is best reduced by more aggressive collections vs. liquidation of underperforming loans? With just a KRI in place, a numeric value, a threshold, and a trend, how do you relay the underlying concerns to your management or board?
- Efficiency Ratio — A score of 50% is considered optimal. It means that operational costs are balanced with revenue. But what if there are inefficient processes in place and revenue happens to be favorable? In good times, does this ratio really drive operational efficiency improvements that might lead to revenue growth and business resilience? In bad times, does it indicate the need to enact a knee-jerk reaction to cut resources where other steps taken may have avoided this need?
- # Customer Complaints Related to Fraud — This is a good measurement of your customer experience areas of concern if compared to other complaint types. However, isn’t it too late in the process to rely on this metric to see how well your customer authentication, fraud detection, and payment operational processes are working? At least turn this into a percentage (i.e., % fraud cases reported from external sources) to get a better feeling of how effective your internal processes are in stopping fraud. Also, other dimensions are lost (e.g., which complaints are favorably closed? Which complaints result in the loss of a customer due to a bad experience? On which channel was the fraud most likely successful? For which events are the bank liable and which behoove the bank to provide the customer with education?). Knowing that the complaint is there is only stating the symptom of the problem.
- # Corrected Access Entitlements — Another reactive KRI. How was access either granted improperly or not removed in a timely manner upon employee transfer or termination? How well equipped is the Access Control Team to perform quality checks within the process? How educated are new managers to know their role in communicating role changes? Does this continue to happen in the same departments over time or is it more pervasive? What does this really tell management or the board without more context?
- # Open Issues Related to Technology — Yet another reactive KRI. Each issue is likely different in scope and magnitude. And how do we know that all true issues are identified and logged to make the number accurate? The KRI can be tuned a bit to be more effective (i.e., # Open Issues Rated Over Risk Threshold Related to Technology or perhaps segmented further: # Open Issues Related to Technology Architecture, Access Management, Cybersecurity, Application Development, etc.). Even if the KRI is refined, there are better measurements (i.e., # Legacy Application Environments, Total Cost of Ownership by Department, # Late Access Removal Requests, etc.).
For a KRI to be effective, it is important to be transparent as to the definition and derivation of each metric, how it is intended to be used, and what additional context will be provided. Comparing the values of several KRIs may also help tell a story and pinpoint any root concerns. Historical values and trends may also assist in the understanding of risk expectations (e.g., is risk cyclical? Is the risk being managed well over time, or is something more needed to expedite mitigation?). Lastly, always be open to revising KRIs to meet the current landscape and stakeholder understanding of risks.
KRIs, KPIs, KCIs, Oh My!!!
It can be confusing and subjective. A KPI can also be read as a KRI. A KCI may be all three! A KRI may actually be better served as a KPI (which is often the case).
These are all just metrics that are merely useful within a certain context, time, and place.
It all comes down to this: these are all metrics whose numeric/qualitative values, context, visual representation, and juxtaposition in reporting tell some sort of story. In order to be useful, they all require additional information, historical understanding, and focused conversation.
Many financial metrics are more mature than operational metrics, and more insights can be assumed based on the science that is understood to be behind them.
Don’t get hung up on the difference as many people do. Focus on the context: always as the question, “…for the sake of what?” when describing the metric or leveraging its value in a conversation.
Both Qualitative and Quantitative Analysis Serve a Purpose
KRI reporting is as much an art as a science. There is a place for qualitative analysis as well as quantitative analysis.
Telling the story of the risk landscape is a complex undertaking and requires a base understanding of operations and finance from a “blueprint” perspective. Quantitative numbers support this analysis.
It also requires some creativity to break through the expected results to consider the unknown and the potential threats to your company. A thoughtful qualitative context supports this analysis.
While it would be expensive to prepare for every possible situation, management needs to be aware of the probable and possible to set the appropriate risk tolerance and to support the resources required for risk mitigation to meet that level of tolerance. Sometimes, that means checking the numbers, other times, it means testing the wind direction and speed.
Visuals and Comprehension
Consider the visuals below. Report A gives the basics from last quarter to this quarter with a simple trend indicator. Report B provides more historical information in a format that provides more context and a sense of the magnitude of change over 4 quarters.
Report A:
Report B:
Notice that virtually the same information can be presented in very different ways! The above represents only two general templates. It may be that you present different visuals for different KRIs within your reporting to align better with the context. Regardless, you should be aware of how the visuals can change the perception of the same information and use this to your advantage when building reporting.
It is important to understand the audience and to direct the visuals to them in a way that resonates with them. Be ready to tell your story in a way that supports your presentation materials and increases comprehension.
Depending on your audience, consistency may be preferred, though some variation will likely break up the monotony and solicit more interest. If you are looking for some good feedback, engage your Marketing partners for guidance.
Additionally, have a brief but thoughtful section on your reporting to provide context and status. Be prepared to speak to them about information, guide the reader through your reporting, and answer any questions you may get.
Conclusion
Key Risk Indicators (KRIs) are merely metrics; tools to convey the generalized direction of risk concern, but they do not tell the whole story without context.
Throwing together a short list of KRIs to present to the board and management is a great starting point. You should certainly address the action being taken when a KRI is approaching or is triggered over the risk threshold.
However, if you do not take the time to set the context and provide supporting information (how the KRI is measured, what other mitigating controls are in place, what other impacts might be in play, etc.), they only serve to check a box.
Provide additional information around emerging risks that may not yet be measured. Use qualitative statements to explain the full risk landscape. Balance the message to be received well by your audience. Provide a thoughtful mix of qualitative and quantitative metrics as appropriate. Maximize the visuals for the personality of your audience. Start a real dialogue with your audience without overwhelming them with jargon. Let them ask the questions. Be direct and informative with your answers. This is how you manage risk and build trust.
