What is the difference between KRIs, KPIs, and KQIs in Risk-based Monitoring? Explained simply.

Anyone familiar with Risk-based Monitoring (RbM), will at some point have come in contact with such terms as Key Risk Indicators (KRIs), Key Performance Indicators (KPIs) and even Key Quality Indicators(KQI).

I’m sure most people are aware that there are differences between these indicators, but these differences are not always obvious. Let me try to clarify.

For the sake of simplicity, let’s take an example: Imagine you’re sitting in your car and you’re looking at the dashboard. In front of you are some KRIs and KPIs.

Car dashboard contains a number of KRIs and KPIs

Car dashboard contains a number of KRIs and KPIs

For instance, a speedometer is a simple performance indicator, it tells you how fast you are driving. It does not tell you if you should drive 50 km/h or 80 km/h in a particular area, this information should already be known to the driver.

A risk indicator is a slightly more complicated construct. In our example, it would be a fuel gauge. It consists of a performance indicator: the number of liters in the tank and additional thresholds, which warn a driver (in two stages) to top up the tank soon. If she/he does not take any action, a complication (or ‘risk’) will occur, in this case, the car will run out of fuel and will break down. 

Thus, this would mean a performance indicator contains only measurement of one metric, and a KRI incorporates additional information about thresholds and certain risk, which may happen if no further action is undertaken.

Typical KPIs within RbM are:

  • Number of enrolled patients
  • Number of adverse events (AEs) per site
  • Number of patient weeks (i.e. how many weeks patients are in a clinical trial)

Now, if we add additional thresholds, or rather, information from experience, we can covert the KPIs into KRIs:

  • Number of enrolled patients is less than it was planned
  • Number of AEs is too high per site comparing to other sites in the same clinical trial
  • Average number of patient weeks per month is decreasing, e.g. due to high drop-out rates
Key Risk Indicator Development

Key Risk Indicator Development, Oncology Phase III Trial (EarlyBird RbM System)

Now it becomes possible to monitor the KRIs and observe their dynamics as shown in the graph above.

Up to this point, things have been fairly simple, now, how about the KQIs?

What can these “monsters” indicate? As the name suggests “Key Quality Indicators” measure quality and quality is a degree of excellence. And it depends on how it was planned.

Despite the significant amount of research, operational cost, and complexity in conducting clinical trials, at its conclusion, a trial is represented by its Dataset.

The Datasets are expected to be in full compliance with protocol specifications whilst also carrying high data quality standards. With this in mind, here we’ll consider KQIs for clinical data.

For the KQI calculation researchers use certain data categories:

    • Data which should not change (longitudinal)
    • Data which should not be cleaned
    • Data which should not be missing (e.g. recorded by devices)
    • Data which is derived from other data
      • E.g. best values in Spirometry
    • Datasets which should correlate with each other (so-called “data integrity”)
      • E.g. questions describing the same physical condition
      • E.g. values based on measurements (e.g. in Spirometry or ECG)
    • Data which has defined value ranges
      • E.g. questionnaires with max possible answers
      • E.g. Spiro and ECG parameters
      • E.g. birthdays not older than 120 years, not in the future
      • E.g. height and weight limitations
      • E.g. patient visits on unusual dates or times (e.g. on Sundays in a country, where Sunday belongs to weekend)
    • Data with different “encoding”/formatting, sanity checks
      • E.g. Wrong entries from person (initials with three characters)
      • E.g. Decimal formatting
    • Data which has time or duration constraints
      • E.g. several visits for one patient on the same day
      • E.g. visits of different patients in unusual short intervals
    • Artifacts in data
      • E.g. wrong Spirometry measurements (either analyzed by ourselves or based on overreads)
      • E.g. impedance of ECG electrodes
      • E.g. artifacts in ECG measurements
      • E.g. missing pages or whole questionnaires
      • E.g. calibration

Longitudinal Categories:

(Either measured per export or maybe per data cleaning over a period of time)

  • Data which should not change
    • Inclusion criteria
    • Raw data
    • Questionnaire data
  • Data which should change based on other changes

These quality parameters can be measured by KQIs separately or united into joined indicators, where several checks can be combined together.

Key Quality Indicators (example of a Phase III Oncology clinical trial, EarlyBird)

Key Quality Indicators (example of a Phase III Oncology clinical trial, EarlyBird)

Above on the picture, you can see a number of joined KQIs. E.g., Vital signs score applies several sanity checks on vital signs such as body temperature, pulse rate, blood pressure.

Another well-known data quality indicator, which is applicable for the whole trial, is “confidence intervals” of the measured data. A threshold used in this context – “statistical significance”, the p-value, which in clinical trials must be less than 5%. Different significance levels (e.g., 1%) may be applied depending on the field of study[1].

Another typical KQI in RbM would be “normality test”, which indicates whether data is normally distributed or skewed. The skewed data is often influenced by a systematic error, which shifts the bell-shaped distribution to one side or another (see below).

Skewed data

The advantage of KPIs, and KQIs is their comparability among trials and retrospectively. The KRIs are more useful for their predictability of the dangerous situations as taking the right actions at the right time can prevent unwanted consequences.

CAPA on practiceCAPA on practice

Summing up, all discussed instruments like KPIs, KRIs, KQIs in RbM aim at identifying, assessing, controlling, communicating and reviewing risks associated with your clinical trial during its lifecycle. This will facilitate making your trial safer for the patients, giving your trial management team full control over a trial and helping to flag any dangerous situations before they occur.

How you negotiate these situations, is up to you to decide.

[1] Sproull, Natalie L. (2002). “Hypothesis testing”. Handbook of Research Methods: A Guide for Practitioners and Students in the Social Science (2nd ed.). Lanham, MD: Scarecrow Press, Inc. pp. 49–64. ISBN 0-810-84486-9.

Interested in learning more about Cyntegrity and our services?

Contact Us

Explore Risk Management Tools
By |2018-03-27T14:29:11+00:00February 18, 2015|Blog|4 Comments

About the Author:

Professional in the integration of data-driven Risk-based Monitoring (RbM) process in international clinical trials of pharmacology. Speaker at regional and global conferences such as: DIA, PharmaForum, PharmaDay, DGGF, etc. 10+ years of experience in data quality projects and biostatistics for the pharmaceutical industry. Life passion: improving clinical research with RbM, driving the RbM research to new frontiers for CROs, pharma and biotech companies.

4 Comments

  1. Uri February 20, 2015 at 3:51 am

    This kind of process could be use in a research
    site

  2. Artem Andrianov February 26, 2015 at 4:18 pm

    Do you mean the application of KRIs, KPIs, KQI for measurement?
    Sure, I even hope that it is used in a research site as a part of holistic RbM process. Today the sites are usually not involved into the RbM process and cannot control their quality and performance.
    In the trials, which are built with our RbM system(See EarlyBird) we are including Sites into the whole process and let them monitor their KRIs, KPIs, KQI regular and even compare with other sites.

  3. Ed Gregory July 2, 2015 at 5:50 pm

    Very nice summary/introduction to KQIs. Succinct. Thanks!

  4. Artem Andrianov July 6, 2015 at 1:36 pm

    Thank you Ed for your kind words!

Leave A Comment

Upcoming Events

  1. Frontiers Health 2018

    November 15 - November 16
  2. PCT Europe 2018

    November 28 - November 29

Register Today!

Get Free Test Access