. After all, an athlete does not need measuring devices and big data computing to perform a double flip and land on his toes; just training and skills. Why would a manager operate differently?
Sometimes though, our intuition can be tricked or overwhelmed by the reality of our work; the athlete can trip and fall. Data, if used wisely, can support intuition, strengthen our confidence; refine our judgment. Airplane pilots may be screened for their skills, sound judgment and physical health; they still rely on checklists and an array of sophisticated instruments to fly their planes because they have the responsibility of success.
Well, what is success?
Well, let’s start at the beginning. What is success? How can we know that we are successful and when can we say that he have succeeded in something? One of the scholar definitions is the following:
A good strategy implies setting goals, planning and executing the plan. According to the Balanced Scorecard methodology, strategy must cover at least four dimensions: People, Processes, Customers and Finance. Goals, plans, targets, thresholds should be set along these for dimensions to ensure a holistic strategic view. Performance is defined as obtaining the right things, in the right way and at the optimal cost. Finally risk is a potential damage resulting from the combination of a threat (external) with a vulnerability (internal).
So in short, what you want is to set your strategic goals intelligently, plan and execute your strategy. Along the way you want to make sure that you obtain the results that you planned for while respecting rules and regulations and sticking to your budget. You also need to regularly assess the external threats and minimize your vulnerability accordingly.
Does your company have a clear clinical strategy? Has it been communicated to you in a way where you understand it and you know your part in it? In industry surveys the vast majority of companies have a clearly communicated strategy (or so does management think) but only 14% of the employees declare knowing it. This is because there is a lack of adequate tracking tools. Business intelligence has been widely developed in recent years but it is limited to gathering information and displaying it in graphs and figures. BI tells you nothing about your strategy and how it is playing out.
Turning data into KPIs
In order to support decision-making, we need to turn information into key performance indicators (KPIs). What this means is that our indicators must describe the three dimensions of performance: results, know-how and means. They must be “3D”.
Results are of course whatever you are trying to achieve. Know-how can be described as qualification, experience and processes. And means can be divided in time, running cost (what we frequently refer as op-ex) and fixed assets cost (what we usually call investment or cap-ex).
Assessing and using risk
Risk is also a major component for our success. We should not fear risk and we should not shy away from studying and understanding it. Along the way, there can be damage to our performance because of external threats (which we cannot control) but only if we are vulnerable to those threats. Of course we cannot take all the possible measures up-front as this would be too costly. Therefore it is important to monitor risk at the appropriate frequency that allows for timely action. One good rule of thumb is to monitor at the immediately lower level from the perceived risk. For example if a risk is quarterly, monitor monthly. If it is monthly, monitor weekly. Finally use risk to determine the relative weight of a KPI.
There are many definitions of risk and many methods of evaluation.
The ISO definition of risk is: “effect of uncertainty on objectives”
The risk-based approach of GCPA is based on the following criteria:
- Critical process and data identification
- Risk identification
- Risk evaluation
- Risk control (mitigation actions)
- Risk communication
- Periodical risk review
- Risk reporting
The analytic part of risk evaluation occurs with the help of the three main risk properties:
- Likelihood that a risk or failure mode materializes
- Impact on subject’s safety, rights and data integrity and reliability
- Detectability – extent to which such threats or errors are detectable
In any case, risk can be used to define what to measure and how often, evaluate the relative weight of each measure, define the appropriate thresholds and corrective actions.
In clinical trial we now have excellent tools for evaluating and monitoring different specific types of risk.
Using the data
We have seen how information can be used to support intuition in decision-making. Key performance indicators are useful for the correct management of conflicting urgencies as well. We constantly deal with urgent situations and we do not always have the capacity to handle all of them at the same time. While the urgency is manifest, in the absence of data it is not always easy to prioritize our actions.
But how should we approach the data that we are able to collect in order to make the best use of them? Business intelligence (BI) tools such as the clinical trial tracking systems (CTTS) and the Risk Monitoring tools are able to collect a very large amount of data. The trouble with these data is that we cannot devote the same amount of work to cleaning them as we do for clinical data. Therefore they are usually incomplete and are met with suspicion.
Small data / Big data
There are two possible paths for making good use of the available data: I will call the first “small data”. This is the traditional approach. Select a limited set of relevant indicators and make sure that all the data is complete and accurate. Ensure full ownership of the information by the relevant department. Data can be easily reviewed in a regular basis and decision made accordingly. Software can be useful to manage this approach but is not necessary. The dashboards and graphs produced by the BI tool are usually sufficient.
I will call the second method “big data” as it is based on a methodology of big data analytics. This will need a dedicated software to be managed and is based on large volumes of data. Information from as many sources as possible are collected and assessed for quality (completeness, consistency with historical values etc.) BI tools are a good source but financial (e.g. SAP) and other databases are also very useful. As long as quality is at an acceptable level (e.g. 70-80%) it can be trusted for the purpose of this method. The exact value of the indicator is irrelevant. Data are assessed using thresholds (to determine what is considered low/medium/high/very high) and tendencies (up or down, crossing a threshold or not). The advantage of this methodology is that it can process data in the same way (by thresholds) at any magnitude (from patient level all the way up to a clinical program) as categories apply similarly at all levels.
Finally, aligning the organization has been a long-standing challenge of any strategy implementation. It is paramount to make sure that all parts of a clinical research unit at all levels are working towards the strategic goals in one or many of the balanced scorecard categories, i.e. people and learning, process excellence, customer satisfaction and financials. In all cases, KPIs must be set in a consistent way throughout the organization. Either manually in the case of a “small data” approach or programmatically in the case of a “big data” solution.
In conclusion, turning data into key performance indicators allows to support intuition in decision making. A robust methodology for defining success, strategy, performance and risk and a multi-dimensional approach are necessary to ensure the efficiency of any performance management system and finally, whether you use a “small data” or a “big data” approach, aligning the organization is paramount to the success of your clinical strategy.
 Jack Welch, John A. Byrne. Jack: Straight from the gut. August 28th 2005, Warner Books
 D. Stamatiadis. Measuring Success: Balanced Scorecard, International Clinical Trials, Samedan publishers, 2011
 How does the New GCP Addendum Influence Clinical Monitoring? October 6, 2015 By Artem Andrianov
 ICH, “Integrated addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2),” 11-Jun-2015. [Online]. Available: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Addendum_Step2.pdf. [Accessed: 16-Jun-2015].
 Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data, Paul Zikopoulos, Chris Eaton, McGraw-Hill Osborne Media ©2011
 Aligning Organizations Through Measurement, Basili, V., Trendowicz, A., Kowalczyk, M., Heidrich, J., Seaman, C., Münch, J., Rombach, D, Springer International Publishing, 2014.