Understanding the Value of Key Risk Indicators – KRIs
William Thomson, the eminent Scotch-Irish physicist (popularly known as “Lord Kelvin”) who determined the correct notation of absolute zero, once said to his students:
“What could be measured could be managed, what could be managed could be done. To measure is to know.”
By rule of nature, we all come across risks and opportunities in our lives. These two are inseparable aspects of the same concept, i.e. the probability of an event to occur that can impact your life either positively or negatively.
Some believe that life is 10% of what happens and 90% of how you react to it. True or not, it tells us that if we want to take control of something we need to pro-actively deal with the risks and opportunities that we are anticipating.
Key Risk Indicators (KRIs) help us measure risks by predicting the surfacing of risks at an early enough stage allowing us to act on them and reduce their impact. They function like sensors that capture risky areas of a process.
The Limited Footprint of Audits
In Pharma, auditing used to be the generally accepted way of dealing with risk management for many years. Today we understand that the pharmaceutical audit is a measure which incorporates several limitations:
- Audits only involve retrospective data and do not evaluate trends
- The random character of quality checks, turning the chance to detect an important and relevant deviation into a lucky shot rather than a targeted hit
Recently the industry is facing a wave of new challenges; the constant increase of research data, protocols becoming more complex, the latest regulatory updates and the game-changing shift to personalized treatments. This rapid evolution has fuelled the need for a reform of the traditional audit approach into a new-style holistic, continuous and automatized one.
A Blind Spot by Design
The old-style quality management approach typically limits itself to random spot-checking of quality risks and non-compliance. In the majority of cases, this detailed spot-checking centers around selected entities only, resulting in a fragmented fact-base. By design, these comprehensive audits leave a blind spot on the systemic quality issues that are out of scope.
This critical process can be significantly improved by continuous, automatized screening of all relevant entities and risk areas. Regular assessments of KRIs, risk identification and prioritization of risk areas greatly enlarge the footprint of the traditional quality management boundaries, since the use of KRIs allows us to capture systemic quality issues and instantaneously adapt our quality assurance measures to the appropriate risk level.
All Relevant Risk Components in Scope
Modern Risk-based Quality Management (RBQM) builds on continuous measurement of defined sets of Key Risk Indicators (KRIs) to prospectively identify and mitigate emerging risks. The selected suite of KRIs measures risks throughout the study conduct whilst simultaneously analyzing the information collected from all aggregated recording systems; e.g. safety information, trial information and clinical data.
Each KRI consists of three main components:
- a metric, which measures a quality property of a clinical trial (e.g., Number of AEs per site, Number of Empty CRF Pages per Site, etc.)
- a threshold which represents an acceptance criteria of the corresponding Quality Property (e.g, a fixed boundary or floating threshold, which continues to adapt the more data becomes available)
- and the risk, which this KRI and its corresponding mitigation actions are supposed to prevent (originating from the risk identification exercise)
KRIs cover the entire conduct of clinical trial process, i.e. from protocol development, site activation, patient recruitment, treatment, study management to eventually patient discharge.
Each phase of a clinical trial is controlled by a focused suite of KRIs flagging for out-of-range deviations and pointing to systemic issues.
After accumulating and processing the information, KRIs can be presented through customized views; e.g. process views, geographical views, and made relevant to specific stakeholder groups such as product and project management.
What should you ask yourselves when you are designing a KRI?
- What do we want to measure and what should these KRIs inform us about?
- What is the link between the risk we measure and the corresponding KRI?
- Where is the data that we need to calculate the KRI coming from?
- Does this KRI inform us about patient safety or data integrity or both?
- What type of measures must my team undertake once the KRI fires?
- What mitigation actions must be applied by my team in order to prevent this from happening in the future?
- What Baseline for the threshold should be chosen and why?
- How often should we review the Baseline?
- Who should be involved in the review of the KRI?
- When does the measure create an action? How much deviation and how many errors do we still accept in the process?
Today’s modern clinical trials management requires a new, holistic and automatized way of checking data integrity and patient safety. Factors like the growing volume of study data, the increasing complexity of protocols, the globalization of clinical trials and the related cultural differences are driving this emerging industry requirement. Distributed data in disparate recording systems form the underlying basis of risk performance and quality management. To utilize this actionable data, a new layer of business intelligence is required that aggregates the information to subsequently drive actions.
As in life, conducting clinical trials is nothing more than risk and opportunity management, and therefore getting an objective picture of a study is prerequisite to executing the right actions. Key Risk Indicators form the core of this widely applicable concept that addresses evolving challenges and provides project managers and QM experts the required objective updates on actionable issues and trends.
In Lord Kelvin’s words; in order to manage quality, quality needs to be measured, and when it is measured, positive improvements will not be long in coming.
