Insights from Cyntegrity’s 2023 AI Summit
Introduction
Cyntegrity’s virtual event, “Responsible AI in Clinical Trials,” in November 2023, brought together speakers from across the industry and beyond, with over 160 attendees discussing the direction of travel, opportunities, and challenges. Keith Dorricott provides a perspective on Cyntegrity’s 2023 AI Summit and why you should book for the next one on October 8 and 9, 2024!
The Rise of AI in Everyday Life
Every day, there seems to be another AI story in the news. Whether it is using AI to recreate the persona of a dearly departed loved one or deep fake voices/images of senior people in your organization that scammers can use to extract money. There are so many possibilities for using AI technology, but lots of ethical questions are being thrown up, too. Just as when the internet was invented, no one really knows how it will end up being used. There is great excitement over the possibilities and also fear of the implications. There is also, perhaps, a nagging suspicion that there is plenty of hype – anyone remember blockchain?
AI in the Mainstream
At the Cyntegrity event, Wole Moses (Microsoft) described how the launch of ChatGPT (and other Large Language Models or LLMs) brought AI into the mainstream and opened eyes to the possibilities. Those who have sat watching an LLM come up with a response to a question often describe it as “magic“. It is so very different from sifting through a list of web pages thrown up by a search engine. Whilst there is plenty of hype, AI is surely with us for the long term and really could have a significant impact in many areas.
AI in Clinical Trials: Opportunities and Challenges
For those in a highly regulated industry such as clinical trials, one of the key questions is whether AI could be part of the answer to the challenge of low productivity while maintaining regulatory compliance and guarding privacy and ethical considerations. AI has been demonstrated in practical use in many areas of clinical trials, such as the “Ask AI” agent for augmented risk assessment. But with so many possibilities of where to use it, where is it best to start? And just because we can, should we?
The Reality of AI Implementation
One of the themes of the event was that AI is not the solution to everything. Unfortunately, “There is no instant pudding,” to quote 20th-century quality guru W. Edwards Deming. Making improvements takes teamwork, time, effort, and thinking. Technology can only get us so far. If a process is failing, it may be best to focus on understanding why the process is failing rather than thinking a new technology is the answer.
Look to fix the process first and then introduce technology where it can add value. Prof Kai Eckert (Mannheim University of Applied Sciences) made this point clearly at the Cyntegrity event when he said that “AI is not Mr. Fix-It!”
Lessons from the Past: eTMF Implementation
I recall the early days of digitizing Trials Master Files (electronic TMF, or eTMF) when I worked at an organization implementing such a technology. Clinical trials involved huge quantities of paper documents and they all had to be managed, filed, and stored in safe facilities for years or even decades. Maintaining an accurate and complete TMF was challenging because it was at a physical location, even though a global trial might be at hundreds of sites. Large quantities of paper were shipped urgently around the world and the TMF never seemed to be accurate or up to date.
eTMF seemed to show huge promise in allowing the project team to access documents from wherever they were based and for documentation to be loaded immediately. Surely complete, accurate TMFs were just around the corner? Sadly, it didn’t work quite like that. IT “delivered” the eTMF ahead of time, but there had been no change management implemented. There were no SOPs, and staff were not trained. And the eTMF was being bolted on at the end of a failing process – all the paper was still going to flow, but it was to be scanned and indexed at the end in a central location. The existing paper process was not working well – to fix it by trying to add a bit of technology was not going to improve it. The whole thing needed a rethink.
Implementing technology in this way can actually lead to a slower process with increased cost and lower quality. Whatever bright, shiny technology is being proposed, it still needs to work in the real world—to help people who are working flat out and have limited time to cope with the new challenges it will undoubtedly bring. These changes are about process and people as much as technology, and the same is surely true with AI.
Starting Small with AI
Another of the themes from the event was that at the current stage of AI development, it is best to start small in a high-risk area such as clinical trials. And to think of AI as an assistant that needs to be supervised. Danny Goh (author of The AI Republic) described the importance of having an “Expert in the loop” such that this expert supervision was always present. And by keeping the scope of AI implementation projects relatively small, they can be used to help demonstrate the real value and to help change minds.
Start small: By keeping AI implementation projects manageable in scope, organizations can better demonstrate real value and facilitate mindset changes.
Lean Sigma and AI: A Parallel
Six Sigma and Lean methodologies were brought into clinical trials from manufacturing in the early 2000s, and many organizations had Lean Sigma programs. Scoping the Lean Sigma projects was always a critical task – sometimes, a “boil the ocean” project would be proposed, and it was important to drill down to understand what the real issues were and what was a manageable scope. Without this step, a new methodology (in the case of Lean Sigma) or a new technology (in the case of AI) is susceptible to hype and ultimately to failure – where the skeptics are proved right. Always better to start small and with a defined scope to demonstrate success. It can grow after that.
Change Management and AI
At two panel discussions – the first chaired by Adam Baumgart (Alcedo Consulting Ltd.), including Francesc Mateu (MongoDB) and Johann Proeve (Cyntegrity), and the second chaired by Nechama Katan (Pfizer), including Artem Andrianov (Cyntegrity) and Silvania Poindexter (Industry Consultant), amongst others, the challenge of change management was discussed from different perspectives. Cyntegrity’s survey showed that the more hands-on people are with AI technology, the more they believe it will have a significant impact on this industry. And so, bringing people into the development process early on is important – for example, with a “sandbox” area for people to try it out, easy ways to feed back concerns and be heard, encouraging a culture of open dialogue and critical thinking. There are plenty of skeptical people in any organization, but when they are involved and listened to and when they begin to see real evidence of improvement, they may begin to change their minds.
Involving stakeholders early and fostering an environment of open dialogue and critical thinking can turn skeptics into advocates. Real evidence of improvement is key to changing minds and driving successful AI adoption.
Regulation and Ethics in AI
The topic of regulation ethics came up many times during the event, with an overview provided by Adam Baumgart of existing regulations in the industry and Moe Alsumidaie (Industry Expert) referencing the “Bill of Rights for AI” published recently by the White House. Meghan Conroy (CaptureProof) discussed some of the ethical challenges for AI in clinical trials and the importance of focusing on transparency, accountability, and health equity. And the challenges of bias in training data sets and careful thinking about data sources and cleanliness came up several times.
The importance of keeping the “Expert in the Loop” was raised in the event many times.
Explore More About Responsible AI in Clinical Trials
About the Author
Keith has been a qualified Lean Six Sigma Master Black Belt since 2007. He has a solid background in the practical use of process improvement, business process re-engineering, and metrics in clinical trials. Having a wealth of experience across processes in clinical trials and being a frequent speaker at conferences such as SCOPE and the MCC Summit, Keith helps make the complex understandable. In his spare time, he also facilitates and provides thought leadership for workgroups at the MCC, such as Centralized Monitoring, eCOA, and Vendor Oversight.
