Register Today!

Get Free Test Access

What is the difference between KRIs, KPIs, and KQIs in Risk-based Monitoring? Simple explained.

What is the difference between KRIs, KPIs, and KQIs in Risk-based Monitoring? Simple explained.

 

Everybody, who came into touch with Risk-based Monitoring (RbM), met at least once such terms as Key Risk Indicators (KRIs), Key Performance Indicators (KPIs) and even Key Quality Indicators(KQI). And I am sure that you feel the difference among them, although the explanation is not obvious. Let me try to clarify these terms.

For the sake of simplicity, let us take an example: imagine you sit into a car and observe your well-known dashboard in front of each driver. Here you can see some KRIs and KPIs.

Car dashboard contains a number of KRIs and KPIs

Car dashboard contains a number of KRIs and KPIs

For instance, a speedometer is a simple performance indicator, it tells you how fast you are driving. It does not tell you if you should drive 50 km/h or 80 km/h in this area, this information a driver should know.

Risk indicator is a bit more complicated construct, in our example it would be a fuel gauge. It consists of a performance indicator –  amount of liters in a tank – and additional thresholds, which warns a driver (in two stages) to tank in a near future. If she/he does not take any actions, a complication (read: risk) will happen, e.g. you will stay without fuel in the middle of a road. 

Thus, this would mean a performance indicator contains only measurement of one metric, and a KRI incorporates additional information about thresholds and certain risk, which may happen if no further action is undertaken.

Typical KPIs within RbM are:

  • Number of enrolled patients
  • Number of adverse events (AEs) per site
  • Number of patient weeks (i.e. how many weeks patients are in a clinical trial)

Now if would add additional thresholds, or better to say, information from experience, we can covert the KPIs into KRIs, here you are:

  • Number of enrolled patients is less than it was planned
  • Number of AEs is too high per site comparing to other sites in the same clinical trial
  • Average number of patient weeks per month is decreasing, e.g. due to high drop-out rates
Key Risk Indicator Development

Key Risk Indicator Development, Oncology Phase III Trial (EarlyBird RbM System)

Now you can monitor the KRIs and see their dynamics as shown on the graph above.

Until now it was fairy simple, now, how about the KQIs? What can these “monsters” indicate? As it is clear from the name “Key Quality Indicators” measures Quality. And quality is a degree of excellence. And it depends on how it was planed.

Despite the significant amount of research, operational cost, and complexity in conducting clinical trials, at the end of a trial is represented by its Dataset. The Datasets are  expected to be in full compliance with protocol specifications while also carrying high data quality standards. Therefore, we will speak here mostly about KQIs for clinical data.

For the KQI calculation researchers use certain data categories:

    • Data which should not change (longitudinal)
    • Data which should not be cleaned
    • Data which should not be missing (e.g. recorded by devices)
    • Data which is derived from other data
      • E.g. best values in Spirometry
    • Datasets which should correlate with each other (so called “data integrity”)
      • E.g. questions describing the same physical condition
      • E.g. values based on measurements (e.g. in Spirometry or ECG)
    • Data which has defined value ranges
      • E.g. questionnaires with max possible answers
      • E.g. Spiro and ECG parameters
      • E.g. birthdays not older than 120 years, not in the future
      • E.g. height and weight limitations
      • E.g. patient visits on unusual dates or times (e.g. on Sundays in a country, where Sunday belongs to weekend)
    • Data with different “encoding”/formatting, sanity checks
      • E.g. Wrong entries from person (initials with three characters)
      • E.g. Decimal formatting
    • Data which has time or duration constraints
      • E.g. several visits for one patient on the same day
      • E.g. visits of different patients in unusual short intervals
    • Artifacts in data
      • E.g. wrong Spirometry measurements (either analysed by ourselves or based on overreads)
      • E.g. impedance of ECG electrodes
      • E.g. artifacts in ECG measurements
      • E.g. missing pages or whole questionnaires
      • E.g. calibration
Longitudinal Categories:
(Either measured per export or maybe per data cleaning in a period of time)
  • Data which should not change
    • Inclusion criteria
    • Raw data
    • Questionnaire data
  • Data which should change based on other changes

These quality parameters can be measured by KQIs separately or united into joined indicators, where several checks can be combined together.

Key Quality Indicators (example of a Phase III Oncology clinical trial, EarlyBird)

Key Quality Indicators (example of a Phase III Oncology clinical trial, EarlyBird)

Above on the picture you can see a number of joined KQIs. E.g. Vital signs score applies a number of sanity checks on vital signs such as body temperature, pulse rate, blood pressure.

Another well-known data quality indicator, which is applicable for the whole trial, is “confidence intervals” of the measured data. A threshold used in this context – “statistical significance”, the  p-value, which in clinical trials must be less than 5%. Different significance levels (e.g., 1%) may be applied depending on the field of study

[1].

Another typical KQI in RbM would be “normality test”, which indicates whether data is normally distributed or skewed. The skewed data is often influenced by a systematic error, which shifts the bell-shape distribution to one side or another (see below).
Skewed data

The advantage of KPIs, and KQIs is their comparability among trials and retrospectively. The KRIs are more useful for their predictability of the dangerous situations as taking the right actions in the right time can prevent unwanted consequences.

CAPA on practiceCAPA on practice

Summing up, all discussed instruments like KPIs, KRIs, KQIs in RbM aim at identifying, assessing, controlling, communicating and reviewing risks associated with your clinical trial during its lifecycle, i.e. make your trial safer for the patients, give your trial management team full control over a trial and inform about dangerous situation approaching. What you will do in these situations, is up to you to decide.

[1] Sproull, Natalie L. (2002). “Hypothesis testing”. Handbook of Research Methods: A Guide for Practitioners and Students in the Social Science (2nd ed.). Lanham, MD: Scarecrow Press, Inc. pp. 49–64. ISBN 0-810-84486-9.

Interested in learning more about Cyntegrity and our services?

Contact Us

By | 2016-11-16T18:35:47+00:00 February 18, 2015|Blog|4 Comments

About the Author:

Professional in the integration of data-driven Risk-based Monitoring (RbM) process in international clinical trials of pharmacology. Speaker at regional and global conferences such as: DIA, PharmaForum, PharmaDay, DGGF, etc. 10+ years of experience in data quality projects and biostatistics for the pharmaceutical industry. Life passion: improving clinical research with RbM, driving the RbM research to new frontiers for CROs, pharma and biotech companies.

4 Comments

  1. Uri February 20, 2015 at 3:51 am

    This kind of process could be use in a research
    site

  2. Artem Andrianov February 26, 2015 at 4:18 pm

    Do you mean the application of KRIs, KPIs, KQI for measurement?
    Sure, I even hope that it is used in a research site as a part of holistic RbM process. Today the sites are usually not involved into the RbM process and cannot control their quality and performance.
    In the trials, which are built with our RbM system(See EarlyBird) we are including Sites into the whole process and let them monitor their KRIs, KPIs, KQI regular and even compare with other sites.

  3. Ed Gregory July 2, 2015 at 5:50 pm

    Very nice summary/introduction to KQIs. Succinct. Thanks!

  4. Artem Andrianov July 6, 2015 at 1:36 pm

    Thank you Ed for your kind words!

Leave A Comment