What is the difference between KRIs, KPIs, and KQIs in Risk-based Monitoring? Explained simply.

Car dashboard contains a number of KRIs and KPIs

For instance, a speedometer is a simple performance indicator, it tells you how fast you are driving. It does not tell you if you should drive 50 km/h or 80 km/h in a particular area, this information should already be known to the driver.

A risk indicator is a slightly more complicated construct. In our example, it would be a fuel gauge. It consists of a performance indicator: the number of liters in the tank and additional thresholds, which warn a driver (in two stages) to top up the tank soon. If she/he does not take any action, a complication (or ‘risk’) will occur, in this case, the car will run out of fuel and will break down. 

Thus, this would mean a performance indicator contains only measurement of one metric, and a KRI incorporates additional information about thresholds and certain risk, which may happen if no further action is undertaken.

Typical KPIs within RbM are:

• Number of enrolled patients
• Number of adverse events (AEs) per site
• Number of patient weeks (i.e. how many weeks patients are in a clinical trial)

Now, if we add additional thresholds, or rather, information from experience, we can covert the KPIs into KRIs:

• Number of enrolled patients is less than it was planned
• Number of AEs is too high per site comparing to other sites in the same clinical trial
• Average number of patient weeks per month is decreasing, e.g. due to high drop-out rates

Key Risk Indicator Development, Oncology Phase III Trial (EarlyBird RbM System)

Now it becomes possible to monitor the KRIs and observe their dynamics as shown in the graph above.

Up to this point, things have been fairly simple, now, how about the KQIs?

What can these “monsters” indicate? As the name suggests “Key Quality Indicators” measure quality and quality is a degree of excellence. And it depends on how it was planned.

Despite the significant amount of research, operational cost, and complexity in conducting clinical trials, at its conclusion, a trial is represented by its Dataset.

The Datasets are expected to be in full compliance with protocol specifications whilst also carrying high data quality standards. With this in mind, here we’ll consider KQIs for clinical data.

For the KQI calculation researchers use certain data categories:

• • Data which should not change (longitudinal)
  • Data which should not be cleaned
  • Data which should not be missing (e.g. recorded by devices)
  • Data which is derived from other data
    • E.g. best values in Spirometry
  • Datasets which should correlate with each other (so-called “data integrity”)
    • E.g. questions describing the same physical condition
    • E.g. values based on measurements (e.g. in Spirometry or ECG)
  • Data which has defined value ranges
    • E.g. questionnaires with max possible answers
    • E.g. Spiro and ECG parameters
    • E.g. birthdays not older than 120 years, not in the future
    • E.g. height and weight limitations
    • E.g. patient visits on unusual dates or times (e.g. on Sundays in a country, where Sunday belongs to weekend)
  • Data with different “encoding”/formatting, sanity checks
    • E.g. Wrong entries from person (initials with three characters)
    • E.g. Decimal formatting
  • Data which has time or duration constraints
    • E.g. several visits for one patient on the same day
    • E.g. visits of different patients in unusual short intervals
  • Artifacts in data
    • E.g. wrong Spirometry measurements (either analyzed by ourselves or based on overreads)
    • E.g. impedance of ECG electrodes
    • E.g. artifacts in ECG measurements
    • E.g. missing pages or whole questionnaires
    • E.g. calibration

Longitudinal Categories:

(Either measured per export or maybe per data cleaning over a period of time)

• Data which should not change
  • Inclusion criteria
  • Raw data
  • Questionnaire data
• Data which should change based on other changes

These quality parameters can be measured by KQIs separately or united into joined indicators, where several checks can be combined together.

Key Quality Indicators (example of a Phase III Oncology clinical trial, EarlyBird)

Above on the picture, you can see a number of joined KQIs. E.g., Vital signs score applies several sanity checks on vital signs such as body temperature, pulse rate, blood pressure.

Another well-known data quality indicator, which is applicable for the whole trial, is “confidence intervals” of the measured data. A threshold used in this context – “statistical significance”, the p-value, which in clinical trials must be less than 5%. Different significance levels (e.g., 1%) may be applied depending on the field of study[1].

Another typical KQI in RbM would be “normality test”, which indicates whether data is normally distributed or skewed. The skewed data is often influenced by a systematic error, which shifts the bell-shaped distribution to one side or another (see below).

The advantage of KPIs, and KQIs is their comparability among trials and retrospectively. The KRIs are more useful for their predictability of the dangerous situations as taking the right actions at the right time can prevent unwanted consequences.

CAPA on practice

Summing up, all discussed instruments like KPIs, KRIs, KQIs in RbM aim at identifying, assessing, controlling, communicating and reviewing risks associated with your clinical trial during its lifecycle. This will facilitate making your trial safer for the patients, giving your trial management team full control over a trial and helping to flag any dangerous situations before they occur.

How you negotiate these situations, is up to you to decide.

[1] Sproull, Natalie L. (2002). “Hypothesis testing”. Handbook of Research Methods: A Guide for Practitioners and Students in the Social Science (2nd ed.). Lanham, MD: Scarecrow Press, Inc. pp. 49–64. ISBN 0-810-84486-9.

Interested in learning more about Cyntegrity and our services?

Contact Us