Adverse Event Reporting - Risk Assessment Process Example

Getting the Risk-Based Process Started in 4 Steps

Do you find the prospect of Risk Analysis and Risk-Based Quality Management (RBQM) to be a daunting prospect? Then bear with me. I shall help you simplify and demystify the process by using an example to illustrate the key steps to consider for the first part of the process; identifying risks, Key Risk Indicators, thresholds, and Quality Tolerance Limits.

Adverse Event (AE) reporting is a critical part of clinical research in terms of participant safety and data quality. Some form of safety assessment should be a primary endpoint in any clinical trial, and therefore AE reporting seems a good choice for this exercise. The examples I give below are not intended to be exhaustive; they are provided for illustrative purposes only.

ICH E8(R1) describes “Signal Detection and Safety Reporting” as the Critical to Quality Factors in Adverse Event reporting.

Step 1 – Critical to Quality Factors

The first step in any risk-based process should be to identify the Critical to Quality (CtQ) factors along with the critical data and processes for the trial. The draft version of ICH E8(R1) describes “Signal Detection and Safety Reporting” as the Critical to Quality Factors in Adverse Event reporting. So, when identifying the CtQ factors, there are two aspects you need to consider here:

The sponsor’s responsibility for an ongoing safety review of the product – signal detection
The requirements and risks in relation to AE reporting

Now, what are the critical data and processes concerning reporting and signal detection? I would suggest the following as a starting point:

CRITICAL DATA	CRITICAL PROCESSES
Adverse Event (AE) reports	Site training in AE reporting Monitoring of AEs Case Report Form completion Data Queries raised and responded to
Serious Adverse Event (SAE) reports	SAE reporting process
Attributability of Adverse Events	Investigator Assessment of causality
Expectedness assessment	Reference Safety Information (RSI)
Suspected Unexpected Serious Adverse Reaction (SUSAR) reports	Pharmacovigilance SUSAR reporting
	Signal detection and escalation
	Safety review – Risk/Benefit assessment
	Annual Safety Report (Development Safety Update Report)

Step 2 – Risks

Based on the table above, you then need to identify the risks at an organizational, trial, and site level to establish the Quality Tolerance Limits (QTLs) and the Key Risk Indicators (KRIs).

CRITICAL DATA/PROCESS	RISKS
Adverse Event Reports	Events misreported i.e. AEs reported instead of SAEs Too many AEs reported Too few AEs reported
SAE reports	Events misclassified Investigator reports SAEs late SAE reports lost in transit Follow up information missing Too many SAEs Too few SAEs
Attributability Assessment	Investigator does not complete attributability Sponsor disagrees with Investigator Assessment Attributability assessment is wrong
Expectedness Assessment	Expectedness assessment not done Assessment uses incorrect version of RSI
SUSAR reports	SUSARs misclassified SUSAR reported late Follow up information missing
Signal detection and escalation	Signal missed Signal identified but not escalated
Safety Review – Risk/Benefit Assessment	Safety review not done Safety review outcome is wrong Actions arising from safety review not implemented
Annual Report	Missing data Incorrect timeline or reporting period Report submitted late

Followed by an evaluation of the risks based on their likelihood, impact, and detectability, so that you can give a risk category for each one. You can use tools available for this process, such as Cyntegrity’s MyRBQM® Portal, which will provide a score for each risk. Those risks with high scores are then the ones to focus on regarding Key Risk Indicators along with risk management and mitigation plans.

Step 3 – Key Risk Indicators

For each of the risks identified, the next step is to establish the KRIs and Thresholds – the metrics used to monitor the risks and the thresholds which should initiate action. The exact metrics will depend on your product and therapeutic area and should be established by statistical methods using available data and information – the examples below are for illustrative purposes only.

As my example here, I will take the first area – AE reporting.

RISK	KRI EXAMPLE	THRESHOLD EXAMPLE
Events misreported e.g., AEs reported instead of SAEs	Data queries raised on AE “diagnosis” field	Data query raised on more than 10% of AE diagnosis fields
Too many AEs reported	Number of AEs reported	More than 2 events per participant
Too few AEs reported	Number of AEs reported	Less than 1 event per 5 participants
		[Thresholds could be set at trial, country, or site level]

Here the number of AEs reported would be an example of a two-sided KRI – it can be too high or too low. Too high might indicate a problem with the product; too low might suggest that investigators are failing to report all adverse events. There may also be geographical differences in AE reporting because of physiological or cultural differences, so you may need to consider this.

Step 4 – Quality Tolerance Limit

Lastly, establish the QTLs at an organizational or trial level. As per our recent article on the top 7 QTLs, I would recommend only one QTL for this area, for example, the number of cases of Adverse Events or Serious Adverse Events of special interest. The specifics of this will depend on your product and therapeutic area, and the parameter used to measure this is likely to be a percentage or number of participants who report the specific AEs or SAEs in question.

Break the Complex Down to Simple

My approach above is a simplified outline of the first part of implementing Risk Based Quality Management. Although the process is complex and technical, you can break it down into simple steps, and once the first round of risk assessment is done, you can carry much of the work forward into the next iteration, which means that each time you do a Risk-Based Quality Management Plan, it becomes easier.