The Cost of Managing Adverse Events Can be High

Adverse Event (AE) reporting, identified by the FDA as one of the most important challenges in clinical research, is known to be prone to under and over-reporting. When starting a project and considering the various ways available to capture the adverse event data, it’s important to ask yourself some analytical questions that help you improve AE and Serious Adverse Event (SAE) capturing and reporting.

“Game changing” versus “there are a lot of unknowns”, were the mixed responses amongst clinicians and other experts to the recently introduced Apple watch 4. The accompanying, FDA cleared software with advanced heart-monitoring capabilities “could increase the chance of false positives” various cardiologists remarked. Some even raised the concern that “evidence suggests the potential costs could actually outweigh the proposed benefits”. Just one of the many real-life examples underlining the need for early and strategic, in this case medical device-related, adverse event planning and management.

Adverse Event: Any untoward medical occurrence in a patient or clinical investigation subject administered a pharmaceutical product and which does not necessarily have a causal relationship with this treatment. An AE can therefore be any unfavorable and unintended sign (including an abnormal laboratory finding), symptom, or disease temporally associated with the use of a medicinal (investigational) product, whether or not related to the medicinal (investigational) product.

Patient Safety First

Analysis of the safety profile is the main objective of running clinical trials, besides demonstrating that the drug works in the specified patient population.

Safety data are being captured during the trial via vital signs, ECG data, and of course laboratory data such as hematology data, clinical chemistry data and urinalysis data. Besides that, (serious) adverse events data are collected with several related items, such as start and stop date of the AE, reason for seriousness, relationship to study drug, action taken and outcome.

Seven Questions to Ask Before You Start

When preparing a project and considering the various ways available to capture the adverse events data, you firstly have to ask yourself the following questions:

Is the investigator supposed to ask the patient generically for any adverse events observed (more or less unsolicited reporting) or will the investigator have a list of predefined potential adverse event terms and is the investigator supposed to read those to the patient one by one?
Is the expectation to mainly get diagnoses reported as adverse events or is the site expected to also document all signs and symptoms related to the diagnosis?
Are the trials planned to be conducted globally or mainly in one region and what might be the impact of such a decision on the reporting of adverse events?
Are abnormal lab values to be recorded as adverse events as well or will abnormal lab values be analyzed separately and reported separately from the adverse events data?
Will the adverse events data be captured in a longitudinal way or per visit or per cycle (in e.g. Oncology trials) and how is that going to influence the data management and the statistical analysis?
Is it planned to have particular adverse events being reported separately from the adverse events page because more related data are planned to be captured – and this is to happen mid-stream of the drug development?
How are cases being handled where an investigator reports an obvious serious adverse event as a non-serious adverse event?

The Pros and Cons of Your Choices

In the following paragraphs I shall address the pros and cons of each of the above decisions on the ‘under-reporting’ and ‘over-reporting’ processes.

Ad 1. Unsolicited reporting versus solicited

When comparing these two approaches for identification of adverse events during the course of a trial (and the same is probably true also for any such questions outside of clinical trials) you will observe a much higher rate of adverse events when a site reads a list of potential adverse events to the patient. This is in particular true when using a list of rather common adverse events such as headache, fatigue, meteorism, hard/soft stool etc. In essence, once an approach has been decided upon for a project there is no option to change to the other one, i.e. from unsolicited reporting to solicited or vice versa since otherwise, the data across studies and in particular for the integrated analysis of safety will not deliver any meaningful results.

Ad 2. Diagnosis versus signs and symptoms documenting

Usually, site staff is being asked to only document an adverse event as a diagnosis, however, in reality, this does not happen. In turn, this causes several challenges during the risk assessment and sometimes also for the final statistical analysis. Adverse events data are being taken as they come in via e.g. the EDC system. Site A may report e.g. ‘gastroenteritis’ (= 1 adverse event) while site B reports ‘gastroenteritis, watery diarrhea, vomiting, stomach pain, cramping, fever, nausea, headache, dizziness’ (= 9 adverse events). Is site A a candidate for the bucket ‘under-reporting site’ or site B one for the category ‘over-reporting site’? In my opinion, both sites are doing ok, however, when looking at the risk assessment and the statistical analysis one probably needs to consolidate the data, e.g. via medical coding (MedDRA) and hope for some of the symptoms to fall into one class such as one MedDRA preferred term or at least a MedDRA high level group term.

Ad 3. Geographic culture bias

When conducting trials in several countries or regions, you will observe a different reporting pattern of adverse events, i.e. in Asia Pacific, the overall number of adverse events will be lower compared to Europe (and you can even see a difference between Eastern Europe and Western Europe) or North America. The reason is a cultural one (Asia Pacific) and may also be due to different legal systems (US). In Asia Pacific a patient is likely to only report an adverse event in case it really bothers him/her. Minor inconveniences will hardly make it into the CRF. In Europe, patients have already been trained to report whatever they observed as potential adverse events over a particular period while in the US, preferably everything is being reported, may it be a minor thing or related to the study drug or not or may there be another reasonable explanation for the adverse event.

What does the above mean for a risk assessment? Is e.g. China under-reporting adverse events or is a site in the US over-reporting? What is the ‘true’ adverse event rate and how to detect those that really stand out – in one way or the other?

One way may be an analysis of the adverse event rates per region or per country and those sites in the US that report a very low adverse events rate should be looked at more carefully – though keep in mind the cases described in Ad 2 above. In case of China and Europe, you should attempt to encourage sites to report whatever they can get hold of and even ask their patients to be frank with respect to potential adverse events.

Ad 4. Personal interpretation versus central analysis

Many lab data are being captured during the clinical trials and sometimes the volume of these data exceeds 40% of all data captured on a patient. It is not a surprise that frequently those lab values go above or below the normal range. When are these data points outside of the normal range supposed to be reported as adverse events and in particular how are they supposed to be reported? Is the decision left to the discretion of the investigator – taking into account whether a lab value in his opinion is ‘clinically abnormal’? How is a situation going to be handled where Site A reports all RBC values >10% below lower limit of normal as adverse events while Site B uses a different approach and takes into account the overall patient situation, i.e. underlying illnesses, age, etc and thus only reports those cases in 25% of the patients? Is Site A the good one and Site B is one that under-reports adverse events?

On top of that, how are the sites reporting those cases? Are they reporting ‘RBC decreased’ or ‘Anemia’? In both cases the sites influence the direction of the adverse event reporting and eventually influence an under-reporting or over-reporting.

I personally think that lab data should be analyzed centrally taking into account the lab values, the normal ranges and – if possible – the overall patient constitution.

How can the above been addressed in a risk-based monitoring setting? I think a good analysis would look into the lab values, e.g. via a patient profile or a graph resulting from the central statistical monitoring graph on one hand and into the distribution of MedDRA coded data in order to identify discrepancies or anomalies in the reporting habit. Once detected early on, the sponsor may still be able to correct or at least modify the various ways of reporting abnormal lab values.

Ad 5. Frequency of data capture

Adverse events could be captured in various ways, all of them eventually influencing the frequency of adverse events.

Usually, the recommended way of capturing adverse events (like also concomitant medications) is a longitudinal one. That means, once an adverse event started, an entry is being done of the eCRF describing the adverse event, as well as a start date, a code for seriousness and so on. Once such an adverse event ended, a stop date is being entered and a code for the outcome of the adverse event. Should for example the seriousness change, a new entry of the same adverse event has to be made with a new start date and eventually with a new stop date. This is the most efficient way to capture those data. Once the data had been coded, the concatenation of the data via the MedDRA code can be done and the course of the adverse event been shown in a graph as an example.

Should one decide to capture the adverse event data by visit, i.e. every new visit a new entry of the adverse event is being done, capturing the data between the previous visit and the current visit, the likelihood of new terminology being used from one visit to the next increases rapidly. For example, on visit one the patient reports migraine-like headache, in the second visit frontal headache and in the third visit simply headache, one should wonder whether these are three different adverse events or are all the same just using a different terminology for the description. Such a reporting habit could result in an over-reporting of adverse events.

The third version of capturing adverse event data is usually being applied in oncology trials and in particular those that are being conducted under a cycle regimen. Depending on the protocol set up, a cycle could be 4 weeks long, could be 6 weeks long or even 8 weeks long. During these cycles, only the ‘worst’ grading of the adverse event using the CTC terminology is being entered, even if that only happens to be the case for a couple of days within a 6 weeks cycle and the adverse event being present for the entire cycle. For example, a patient develops a hand-foot syndrome after being in a trial for a week, the syndrome is really severe and may even be serious for a couple of days. The syndrome is being treated with adequate medication and after another few days, the syndrome gets much better, however, persists as a non-serious, mild hand-foot syndrome. The entry on the case report form, however, is 5 weeks of serious hand-foot syndrome, which truly is an overreporting of an adverse event, since in fact it was only serious for a week.

In summary, the way how adverse event data are being captured impacts the frequency and may be even the seriousness in one way or the other. Thus, the longitudinal capturing of adverse events is probably the best approach, since it permits a break down into e.g. cycles or any other breakdown after the study completion.

Ad 6. Single AE page versus multiple reporting forms

Sometimes, certain adverse events get more attention after a series of studies had been conducted. For example, you may want to develop a blood glucose lowering drug. All adverse events are being reported on the adverse events page in the CRF.

Half way through the development of the compound, someone in the organization realizes that the hypo- and hyperglycemic events may be candidates for more attention. It might be of interest, what the patient did prior to those events, e.g. exercising, what the patient ate and the details of the meals, and finally also how much insulin the patient injected just prior to these events.

A new form is being created and for all new studies, hypo- and hyperglycemic events are being captured on those forms, and not on the adverse events form anymore.

When analyzing the data for the integrated summary of safety (ISS) the hypoglycemic events in the adverse events tables and graphs will go down to zero, just due to the fact that the data had been captured elsewhere.

In such a case, you have to be very careful on the reporting of the data and fully understand what happened during the drug development and data capture process in particular. Otherwise and under-reporting of adverse events may result.

Ad 7. Serious AE versus non-serious AE

Another case of a potential under-reporting of adverse events, in this case of serious adverse events, is the case, when an investigator reports an obviously serious adverse event as a non-serious adverse event? How can that happen?

The EMA defined a list of about 8200 adverse events MedDRA terms in the so called Important Medical Event list, a list of adverse event terms that are either core serious or serious under certain circumstances. Not every investigator will be able to keep all those cases in mind and may – inadvertently or consciously – decide upon an adverse event being non-serious.

If that happens more frequently in a study or even a drug development program, certain (serious) adverse events may be underreported, since they ‘disappear’ in the two buckets of adverse events, a non-serious bucket and a serious bucket. That means, a sponsor company has to specifically take care of them and ensure that the ‘true’ frequency and seriousness is being reported in the study report and finally also in the integrated summary of safety.

From all of the above scenarios you can easily conclude that planning a drug development, planning a CRF or data capture method, looking at the data that come in house and finally analyzing the adverse event data should be done very carefully by taking all influencing factors into account in order to not under- and not over-report adverse events.