AI In Clinical Trials: Stop Hyping It, Start Proving It
By John Oncea, Chief Editor, Clinical Tech Leader
Every clinical research conference session in the last two years has featured some version of the same claim: AI can do this; AI can do that. David Vulcano, CEO of the Association of Clinical Research Professionals, says he’s tired of hearing it, and he thinks the entire industry should be too.
“I think the world, myself, and my organization are very tired of just hearing how AI can do this and AI can do that,” Vulcano told me. “It’s time to start showing people how to use it, stop telling people all the lofty stuff.”
He should know. About two years ago, he authored ACRP’s responsible-oversight guide for AI in clinical research, and he’s currently leading an industry think-tank editorial on sharing AI successes and failures, expected to be published soon in the DIA journal.
A Call To Publish Failures, Not Just Wins
Vulcano’s central argument is one most of the industry hasn’t fully absorbed: clinical research already has a rigorous model for reporting failure, and it isn’t using it for AI. When a drug trial produces results, the industry doesn’t just publish that the drug worked — it publishes the adverse events, the protocol deviations, the data quality problems. “My call to action was we need, as our industry, to treat AI and the sharing of AI successes – and not-so-successes – with the same diligence and fervor as we do our medical products,” he said.
The example he offered was pointed. An organization builds an algorithm meant to speed up patient recruitment, only to discover it was trained predominantly on Caucasian patient data and performed poorly at identifying non-Caucasian candidates.
That kind of finding, Vulcano argued, should be published as openly as a drug’s side-effect profile, not buried to protect a vendor’s reputation or a sponsor’s competitive position. “Put your competitiveness aside and share your experiences in a manner that helps us all grow,” he said. Without that transparency, he warned, the industry risks competing on “how many patients your AI killed versus my AI killed” instead of building the shared trust the field depends on.
Where The Real Training Gap Is
Vulcano is skeptical of the idea that AI adoption in clinical research is primarily a technology problem. Most research organizations, he said, have moved to some form of walled-garden or sandboxed AI environment, addressing the earlier “Wild West” era of staff pasting confidential protocols into public chatbots. The bigger gap, in his view, is skill: end users are handed large language models with almost no training in how to use them well.
He drew a direct comparison to the early days of internet search. “We’re at that infancy stage with prompt engineering and LLMs with the average user,” he said, comparing today’s research staff to searchers who once had to learn Boolean operators to get useful results from Google. ACRP is leaning heavily into hands-on prompt engineering training this year specifically to close that gap, not because staff lacks access to AI tools, but because they don’t yet know how to get more than a fraction of the value out of them. “I want to help them sharpen a saw,” he said, “so that they don’t have to take 20 prompts to get something they can get in three.”
Agentic AI, in his assessment, is a separate and more advanced conversation. Most research professionals today are working with large language models to become more efficient at existing tasks, not deploying autonomous systems that take action in the real world; that still requires more technical infrastructure, more risk assessment, and a different skill set than most coordinators and monitors currently have or need.
The Governance Friction Nobody Talks About
Even organizations ready to deploy AI responsibly run into a second, less-discussed bottleneck: onboarding friction. Vulcano described organizations fielding as many as a hundred requests a day for new connected technology, each requiring its own cybersecurity review, and AI tools add an additional layer, since committees increasingly want to see bias analysis and data-use terms before approving a new system. A sponsor that swaps vendors two days before a study start, he said, can delay that study’s launch by six to nine months while sites work through their own review processes.
That friction, paradoxically, favors sites that already have AI built into systems they’ve vetted themselves, one more reason, in Vulcano’s view, that sites investing in their own well-governed technology come out ahead. It’s the same site-versus-sponsor tension explored in this series’ companion piece, Duplicate Tech, Not Technology, Is Burning Out Trial Sites (coming September 3).
What ‘Doing It Well’ Actually Looks Like
Vulcano’s advice to organizations moving too slowly is simple: start now, in low-risk applications. Draft a consent form. Draft a social media post. Build comfort before extending into higher stakes uses. His advice to those worried about AI’s imperfections is equally direct: hallucination isn’t a flaw to wait out; it’s a property of the tool to manage. “It’s my job to train the puppy to do what I want the puppy to do,” he said.
What he won’t accept is more unsubstantiated enthusiasm. The industry doesn’t need another conference panel promising that AI will transform clinical research. It needs organizations willing to say, publicly, what worked, what didn’t, and why; the same standard clinical research already holds itself to for the products it exists to test.